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Final Evaluation Report 

Introduction 

The Champaign County Workplace Literacy Project was funded through the National Workplace Literacy 
program. The general purpose of this federally funded program is to provide demonstration projects that 
teach literacy skills needed in the workplace through exemplary education partnerships between business, 
industry or labor organizations and educational organizations. The Champaign County Project was 
developed to address specific RFP priorities related to training adult workers and specific activities that 
included (d) upgrading/updating basic skills of adult workers in accordance with the changes in workplace 
requirements, technology, product or process; and (e) improving the competency of adult workers in 
speaking, listening, reasoning and problem solving. The four major goals of the project are to: 

GOAL 1: Provide on-site workplace literacy instruction to a minimum of 300 employees of the 
four local businesses involved in quality control process, 

GOAL 2: Provide basic knowledge and skills necessary to participate in TQM/SPC processes, 

GOAL 3: Increase worker productivity leading to improved job maintenance, career advancement 
and decreased turnover, 

GOAL 4: Demonstrate a national rural workplace literacy model that can be replicated and to 
develop and disseminate a work-based curriculum. 

The Champaign County Board of Education, also a member of the County Business Advisory Council, 
established a collaborative relationship with four(4) local companies to carry out the activities described in 
the original proposal. The Workplace Literacy project was designed to focus on the workplace literacy 
training needs of the adult workers of four(4) local companies. A brief Ascription of these companies is 
provided in the table below. 







Pr;oduct($) 


Comdyne, Inc. 


93 


Fiber glass tanks for Natural gas 


The Hall Company 


48 


Custom name plates, 
Membrane switches, 


Grimes Aerospace 
Industries 


850 


Lighting (internal and external 
aeronautics) 


LewiSystems 


149 


Plastic injection molding for recyclable 
containers 



♦Estimated number of employees as described at the beginning of the project. 



The primary purposes of this project were to provide on-site training to company employees based on the 
needs of participants, and to design and implement a workplace curriculum that targets job specific skills 
needed to prepare employees to participate in TQM/SPC processes. 

Purpose of the Extern al Evaluation ^ A 

The requirements of the RFP indicate that the external evaluation should be formative and sur imative and 
must be based on student learning gains, the effects of job advancement, job performance ar d project and 
product spread and transportability. The purpose of this report is to determine the extent to which the 
above stated goals and their related objectives were achieved through the activities of this project. 
This report will provide an objective formative and summate evaluation of the project's activities ai»v. 
outcomes. Sections 1 and 2 of this report will provide a brief descriptive overview of the project. For 
more detail see the project Final Report. Sections 3 and 4 will provide a description of how the evaluation 
was conducted, instruments used and the results of data collected. Section 5 summarizes final conclusions 
drawn from the evaluation report. All materials referenced in the body of this report will appear the 
appendices. 
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Overview of the Project 



The purpose of this chapter is to provide a brief overview and limited chronology of the major activities 
involved in Champaign County Workplace Literacy Project which relate to the data collected through the 
external evaluation process. The intent is provide the reader with a frame of reference for the context and 
process used to carry out project activities such that the evaluation results can be understood within a 
context. For a detail description and explanation of project activities refer to the proceeding sections of 
project final report. 

1 PROGRAM CONTEXT ~l 

The Champaign County Workplace Literacy project operated from the County Board of Education office 
located in Urbana, Ohio. The project was staffed by a project director (1 FTE), a worksite 
coordinator(part time), two(2) teachers(part-time), and 1 highly experienced workplace consultant. The 
project established a Problem Solving Committee comprised of representatives from each of the 
participating companies. The Problem Solving committee provided on-going input to project staff 
regarding the perceived needs of company employees, on-going feedback on how those needs could best 
be met by the project, and their perceptions of the progress of the project As a result of on-going 
communications with the participating companies, a second work group of Foremen/Supervisors also 
evolved. This group provided feedback to project staff regarding their perceptions of the project. In 
addition, this group was able to exchange and share information relevant to their supervisory roles in their 
respective companies. This became recognized as an additional benefit derived from being involved in this 
project. 

The companies that participated in this project are all established entities in the Champaign County 
Community. This rural farm community represents an interesting mixture of agricultural, small family 
owned and operated services, and commercial businesses. Comdyne, Hall Co., Grimes Aerospace and 
LewiSystems represent some of the major employers in the community along with other major services 
such as the local hospital, the School Board, etc. During periods of more economic stability, such 
companies were viewed as a stable source of employment for individuals and their family members. Most 
employees were educated in local city/county schools and many remain in these jobs for the majority of 
their employment careers. As in any organizational structure, each company possessed a unique 
organizational culture influenced by a combination of factors including, but not limited to, the 
organization's mission, the philosophy as implemented by the company executives and supervisors, and 
the influence of a conventional mid-Ohio rural setting. 

Over the course of this project, several of the companies experienced a number of changes due to factors 
including a fluctuation in the economy, the varying demands placed on them for their products, and 
management decision that were be> ond the control of local staff. For example, as a result of down sizing 
and internal reorganization, one company experienced an across the board reduction in force. A number 
of individuals who were eithd involved in the training or served as contacts with project staff were 
terminated. In another instance, one company was bought out. This is relevant in that it effected the 
climate within the participating companies, the attitudes of the employees, and their personal interactions. 

PROGRAM INPUT 

Champaign County Workplace Literacy Project Planning 

The initial project planning team consisted of the project director, work site coordinator and consultant. 
This team obtained input from company representatives to determine global training needs, and engaged in 
problem solving activities to make decisions regarding the process for selection of teachers, selection of 
training participants, to determine a process for scheduling classes and for on-going involvement with the 
participating companies. 

The proiect planning team enlisted the involvement of a company representative to conduct the teacher 
selection process, including screening and interviewing applicants. Three weeks of teacher training and 
orientation was conducted prior to the beginning of the first teaching Cycle in August, 1993. 
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The planning team considered a number of factors related to the changing climate within the participating 
companies in determining criteria for participant selection. The context included the fact that companies 
were now more inclined to compete internationally, were engaged, to some extent in implementing TQM 
and flattening their organizational structure, were placing increased demands on employees related to 
communication skills, problem solving skills, asking better questions, using more effective 
documentation and interpretation of documentation. This type of shift in expectations extends beyond the 
basic knowledge of technical skills that may be learned through vocational/technical training, and basic 
reading or math skills that may be learned in secondary or post secondary training. It involves being able 
to utilize processes for making the skills that employees posses work in the work environment. 

The team established some generic criteria related to the basic skills involved in workplace literacy, 
continuous process improvement, improved communication skills, problem solving skills, and the ability 
to apply those skills to the workplace. The team conducted problem solving committee meetings prior to 
and during the course of the project to maintain ongoing input and feedback to the project. Additional 
feedback was obtained from the foremen/ supervisor work group throughout the course of the project 

Other input that influenced project planning and implementation included data collection activities such as: 

• conducting the literacy task analysis(LA) on the majority of job classes 
represented in the training classes, 

• administration of assessment instruments including the ABLE, TALs, CLOZE, 
GAP, Learning Style profile and inventory 

Project staff developed lessons based on input from the LTA, basic skills information from assessments, 
learning styles data, input from the participants relative to their learning expectations and input from 
foremen/supervisors relative to their perceptions of the participant's learning needs. 

During the planning stages, key personnel met with companies individually to discuss site specific details 
of implementing the classes. This included determining a schedule for conducting the class sessions, 
identifying specific locations for the classes that were conducive to teaching, making work related 
materials available to teachers and project staff, obtaining security clearance, where necessary, for project 
staff and other details. 

Project staff also engaged in a number of training and staff development activities throughout the course of 
the project These including internal training conducted by the project consultant, external training 
opportunities available through Wright State University. 

PROCESS 

Teacher Selection And Training 

Project staff utilized the Champaign County Board of Education procedures to advertise for the teaching 
positions. A limited number of applications were received. It was perceived that this was a result of the 
positions being part time not full time. Applicants were screened and staff were selected based on their 
experiences and expertise. Two individuals were selected, both individuals were from the surrounding 
area and brought different backgrounds and experiences to the project. As the project progressed, it was 
determined that more time was needed to plan and prepare for classes. Therefore the teacher's work time 
was increased. 

Prior to the selection and training of teachers, key personnel were involved in conducting Literacy Task 
Analysis of the various job classifications of potential participants. This input along with other 
information collected from company representatives through discussions and needs assessments was used 
to design teacher training, to develop initial lessons and lesson plans to be carried out during Cycle I. The 
teachers participated in 3 weeks of training between June and August, 1993. The training was conducted 
by the project key personnel and the content focused primarily on metacognition and workplace literacy 
instruction. Limited emphasis was also placed on adult learning/education. One noted limitation of the 
training was due to the time frame within which the teachers were brought on board. Teachers had limited 
opportunities to go on-site or to become familiar with the climate/culture within the companies, to build 
relationships with the potential participants prior to the start of classes. This was addressed during 
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subsequent Cycles. Additional time was allocated for teachers to participate in project planning and to go 
onsite to observe workers in their work environment. 

Participant Selection 

Prior to each cycle the companies made decisions about the specific job classes that would be involved in 
the training. A sample listing of the job classes of individuals who participated in the training by company 
is provided below: 

JOB CLASS INVOLVED IN TRAINING 

♦ Detaining 

♦ Sanding 

♦ Pressure Testing (Valver) 

♦ Ring Winding 

♦ Lay Up 

♦ Cutter 

♦ Assembler 

♦ Screen Printer 

♦ Draft person 

♦ Analyst 

♦ Repair Technician 

♦ Receiving 

♦ Assembler 

♦ Machinist 

♦ Model Maker 

♦ Contract Assistant 

♦ Contract Administrator 

♦ Senior Business Administrator 

♦ Maintenance 

♦ Press Operator 

♦ Molder 

♦ Secondary 

The process of identifying and selecting individuals from these job classes to participate in the training 
each cycle was somewhat complicated. Project staff met with representatives from the problem solving 
committee to made recommendations regarding the process for selection of employees to participate in the 
project During the Cycle I a number of variables appeared to impact how companies made the final 
decision regarding which employees would participate. It is perceived that some factors were related to 
production, the number of job classes they wanted represented in the classes, the impact of teams or 
sections being out of production at the same time, the need to involve management/ supervisory staff in the 
training, etc. The outcome was that companies used different approaches (volunteer, selection, etc.) to 
make decisions about who would participate. In some instances there was a diverse representation of job 
classes involved in the training and in other cases the classes were comprised of limited job classes or 
limited involvement of management/supervisory staff. 

Another issue faced by project staff related to the type of information that selected participants received 
about the purpose of the project, why/how they were selected, why/how test information and class results 
would be used, and how/why they would benefit from participating in the training. There was a lack of 
clarity among many employees who participated in the training particularly during Cycle I regarding what 
they would be learning, how it would be of benefit to their jobs, whether class tests and other data would 
be used to determine lay-offs, etc. During subsequent Cycles II and in, the project staff implemented 
strategies to provide more specific information about the classes to companies. The project staff wanted 
clarify the purpose and benefits of being involved in the classes. 

Implementation Of Training Classes 

The project implemented a training scheduled that involved three(3) cycles of teaching lasting for fifteen 
(15) weeks each. Individuals within each job class selected by the companies participated in one cycle of 
training. As a result, each company involved a different set of participants in each of the three training 
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COMPANY 
Comdyne 



The Hall Company 



Grimes 



LewiSystems 



cycles. One exception to this was the participants from LewiSystems. Since the class at LewiSystems 
met only one day per week. The same group of individual; remained in the training session for two 
consecutive cycles in order to receive fifteen(15) weeks of training. 

The class schedule varied from Cycle I, to II and III. In general the class schedule involved participants in 
a two hour class on two(2) days per week for a total of four(4) hours of class per week for 15 weeks. 



A sample schedule for the two teachers is illustrated in the table below: 



Timp 
X UilCs 


ivivjiiLuiy 


X UC/oUdy 




Thi ircHnv 


Friday 


AM 


9:00: Hall 
Company 


8:00: 

LewiSystems 
Grimes - CBT 


9:00: Hall 
Company 


8:00: Grimes 
CBT 


Pr 


oject Planning 


PM 


1:00: Grimes - 
Operations 


1:00: Comdyne 

1:30: Grimes - 
Product sup. 


1:00: Grimes- 
Operations 


1:00: Comdyne 

1:30: Grimes - 
Product sup. 
4:00: 

LewiSystems 




f 



Training classes were located in a variety of settings ranging from a company conference room to break 
area influenced by noise, to separate classrooms. As time progressed it became apparent that the learning 
environment was very important and companies took appropriate measures to provide at least minimally 
adequate space for the classes. The Hall Company felt that the learning environment was key to the 
training. They took the initiative to make available a separate classroom facility that was more conducive 
to the training. 

The organizational flow of activities was mostly consistent for each cycle. A break period was established 
between Cycle I and Cycle II and between Cycle II and Cycle in to facilitate project planning. Project 
staff used data from assessments, participant surveys, informal discussions, and input from the problem 
solving committee and foremen/supervisor workgroup to inform the planning process, make adjustments 
in the processes used and modify instructional materials. 
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External Evaluation Methodology 



The stated purposes of this external evaluation are 1) to determine the overall effectiveness of the project in 
implementing the proposed goals and objectives; and 2) to determine the extent to which the training had an 
impact on the productivity of participants. 

Evaluation Design 

The basis for the evaluation design used for this external evaluation is the Triphase Evaluation process 
(Johnson, in press) which is based upon Stuffelbeam's CIPP decision Making Model (197 1). The Triphasal 
process which evaluates input, process and outcome focuses on different aspects of the program. During the 
input phase emphasis is placed upon determining the needs of workers in the work environment and the 
needs of employers to influence the development of a training program that will meet those specific identified 
needs. The evaluation should determine whether good decision making rules were utilized. Baseline data is 
established through a pre assessment of worker skills/abilities and existing data on productivity. 

"Hie process phase is designed to determine the level of integration of inputs into procedures that yield the 
appropriate output. It focuses upon monitoring the implementation of project objectives, identifying 
discrepancies between what was planned, what actually happened, and in this case making appropriate 
adjustments to the training prior to implementing final training activities. The input and process phases are 
therefore part of the formative evaluation and provide data for program planning, implementation and 
modification. The outcome phase (summative evaluation) involves evaluating the impact of the program. A 
summary of the focus of each phase of the evaluation is provided in the table below. 



Phase 
Input Phase 



Focus 

• Determine needs of 
participant, employer 

• Determine expectations of 
participant, employer 

• Establish baseline data of 
participant skills, 
productivity 



Process Phase 



Monitor implementation 
of project goals, learner 
goals, 

Obtain feedback from 
participants supervisors, 
project staff regarding 
training, 

Determine the level of 
integration of inputs into 
procedures that yield the 
appropriate output. 

Identifying discrepancies 
between what was planned, 
what actually happened. 

Make recommendations 
regarding modifications, 
adjustments, etc. 



Questions to be addressed 

• Are the project goals/objectives 
congruent with the goals and 
expectations of the participants? 

• Were good criteria used for the 
selection of company employees to 
participate in the training? 

• Were topics/activities of pertinent to 
the specific job requirement of 
participants included in the training? 

• Was input from appropriate sources 
used to inform the implementation 
of the project? 

• Were appropriate information 
inputs used in the management of 
the project? 



Measures used 

• Pre and post training 
survey of participants 
needs/expectations 

• Pre and post training 
survey of supervisor/ 
foremen expectations 

Pre and post measures of 
productivity 



Interviews with project 
key personnel, 

Participant surveys 

Interviews with 
foremen/supervisors 



Were training activities applied 
effectively? 



Were effective management 
procedures used to make decisions? 
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Phase Focus 

Outcome Phase • Evaluation of project 
objectives/ outcomes 

• Evaluation of proj ec t 
effectiveness 

• Make recommendations 



Impact 



What effect were the effects 
of the training? 



Questions to be addressed 

• Were the goals and objectives of the 
project achieved? 

• Did the training/ intervention meet 
the workers' skill development 
needs? 

• Did the training impact the workers 
opportunities for advancement?, 

• What was the worker's perceived 
satisfaction with their involvement 
in the training? 

• What was the perceived satisfaction 
of the foremen/supervisors? 

To what extent did the intervention 
impact the worker's levels of 
productivity? 



Measures used 

• Pre test/posttest 
comparison of 
participants performance 
on the TALs 

- Pre/post participant 
surveys 

Pre/post foremen/ 
supervisor surveys 



Pre training/post 
training comparison of 
productivity data, 
Foremen/supervisor 
surveys 



Procedures for Data Collection 



Qualitative data collection procedures 

The external evaluator met with project staff early in the project to discuss the evaluation porches, to 
identify the types of data to be collected, to determine who would design forms, collect data, types of 
products to be reviewed, and a potential time frame for collecting data during cycles I. Ongoing 
discussions with the project director provided updates on project activities, issues occurring with 
participating companies, and to determine ongoing data collection for cycles II and HI. 

Several data collection instruments were developed over the course of the project to assess project 
implementation activities and to obtain data needed to evaluate the perceived effectiveness of the 
Workplace Literacy Project. These instruments include the following: 



Type of instrument 



Purpose 



Pre and Post training Participant Survey 
Form (see Appendix A) 



Foremen/Supervisors Surveys and 
Interview Forms (see Appendix B) 



to determine participant's perception of their needs, what 
they would gain from the being involved in the training, 
and to determine their perceived satisfaction with the 
training after completion. 

to determine supervisor's perceptions regarding the 
training needs of participants. 

to determine supervisors perceptions of critical skills and 
behaviors of top t average and low performing employees. 

to identify specific critical behaviors observed in 
employees who participated in the training. 

to determine supervisor's perceived satisfaction with the 
training. 
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Interview protocols for Project Key to determine processes and procedures used to carry out 

staff(see Appendix C) the activities of the project 



to determine project staffs perceived strengths and 
weaknesses of the project. 

to solicit staff recommendations of strategies that would 
strengthen the project. 

The participant survey was an informal survey presented to each class by the Project staff. Paiticipants 
were instructed to complete the survey on their own time and return to the teachers. The surveys were to 
be collected by project teachers and returned to the project evaluator. A post training survey was 
developed based on the same or similar questions and was to be administered after the completion of each 
cycle. 

A series questionnaires were designed to conduct cne-to-one interviews with the participating company 
foremen/supervisors, project administrative staff and project teachers. At least one company liaison was 
interviewed from each company. Face to face interviews were also conducted with project administrative 
staff and with project teachers. The company foremen/supervisors discussed their perceptions of the 
usefulness of the workplace literacy training, how it would contribute to improvement in skills and 
behaviors of employees, and it's potential impact on productivity. Project staff interviews focused on the 
project implementation process including start up, needs assessment, teacher training and overall project 
management. The information resulting from these interviews and surveys will be reported here to 
document the effectiveness and impact of the project. 

Quantitative data collection procedures 

The project staff elected to use two standardized instruments to measure progress of students in improving 
overall literacy skills. The Test of Applied Literacy Skills (TALS) has two components which measure 
prose literacy and document literacy on a pre and post basis. 

The Adult Basic Learning Examination (ABLE) is a mathematics test with components which measure 
number operations and problem solving skills also on a pre and post basis. To accommodate the 
heterogeneous educational backgrounds of the participants, the project staff applied three different levels 
of the ABLE test. 

Test Administration Procedures 

Both the TALS and ABLE tests were administered and scored by the project staff. The pre-tests were 
administered during the first week of classes and the post test during the final week. Two sites did not 
collect post test data Grimes at Rt. 55 site did not receive the post test because classes ended earlier than 
scheduled. The class at the LEWISystems site will complete the 60 hours of training at the end of the 
second cycle and will receive their post test at that time. It is important to note that upon completion of 
Cycle I, a number of questions were rained by project staff regarding the use of the ABLE. One problem 
staff encountered related to the color coding by level of the ABLE response forms. Participants seem to 
feel that they were being categorized. Another issue was whether the ABLE measured the skill/content 
being covered in the class. Project staff decided to discontinue use of the ABLE and to use only the TALS 
as a standardized measure for Cycles II and III. Therefore, the results of the able will only be discussed in 
relation to Cycle I participants. 

Statistical Data Analysis 

The statistical analysis of the pre and post tests was done by the project evaluator. The method of analysis 
was a comparison of matched pairs of the pre and post results and a percentage analysis of the observed 
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changed during the period. The paired comparison analyzes significance levels of the changes through the 
use of a t-test at a 95% confidence level. 
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Section IV provides an overall summary of relevant findings and recommendations in relation to project 
planning, needs assessment, project implementation, project management, and test results. Appendix A 
through Appendix D provides a sample instruments used to collect data from each group. Appendix E and 
Appendix F provides the statistical summary tables of the TALS and ABLE.' 

Curriculum Material s Review 

The project evaluator requested copies of instructional materials, record keeping forms and overview of 
the curriculum process used by project staff. The following materials were received at various points in 
the project however the majority were received after Cycle III. 



class syllabus 
Weekly class log form 
Literacy objectives form 
Job skills inventory 
Draft Curriculum Description 



Literacy Task Analysis 
Individual Educational plan form 
Lesson plan format 
Legos exercise 

Draft Workplace literacy Curriculum Outline 



After reviewing the Draft Curriculum and reviewing transcripts from teacher interviews, my assessment of 
the organizational framework used as a basis for instruction of the outlined below: 



OBJECTIVE 



ACTIVITIES 



Orientation and assessment of participant 
needs/ learning goals 



Instruction and application 
Metacognition 
Transition/Practice 
Job Application 



♦ Data collection including: administration 
of pre-test(ABLE Cycle I only) (TALs) 

♦ Participant surveys administered 

♦ Learning styles profiles/inventory 

♦ Discussion of participant 
needs/expectations 

♦ Introduction to metacognitive 
concepts/process, 

♦ Introduction of learning styles concepts 

♦ Process applications (i.e. problem 
solving, cooperative learning teams) 

♦ Incorporation of bash skills instruction 
(reading, math, verbal and written 
communication) 

♦ Application of processes and content to 
job specific activities/projects 

♦ Use of individual/group projects related 
to specific job responsibilities 

♦ Data collection including: administration 
of post-test(ABLE Cycle I only) (TALs 
for cycles H & HI) 

♦ Participant surveys administered 

The sequence of instructional activities as described by project staff was very organized, however staff 
appeared to maintain a level of flexibility based the perceived needs and readiness of project participants. 
Given this framework, the basic skills content and instructional materials included differed from class to 
class. Using assessment data, observations, and other input, teachers were able to determine the basic 
skills content (e.g. math embedded in language, reading, verbal communication, written communication, 
math computation, etc. ) where participants needed to focus. These content areas were incorporated in the 
activities in the class. In addition, teachers indicated that they used a variety of materials including: 
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Type of material 



Examples 



teacher made 



commercial 



workplace materials 



individual and group learning activities 

math and reading activities derived from 

workplace materials 

brain teasers 

group process activities 

blue prints 
shop papers 
routing materials 




Champaign County Workplace Literacy Project 



Final Evaluation Report 



Section 4 



Project Evaluation Results 



i 



m Champaign County Workplace Literacy Project ^ Final Evaluation Report 14 

ERIC 1 ' 



Project Evaluation Results 



The purpose of this section is to describe the results of the evaluation based on quantitative and qualitative 
data collected during the project. This section is organized in three categories Input evaluation, Process 
evaluation and Outcome evaluation listing the major questions of concern and presenting substantiating 
documentation. 

Input Phase 

The focus of the evaluation of this component of the project is to determine whether appropriate input was 
obtained to inform the decisions made to implement the project. The questions to be answered in the input 
evaluation are: 

| Was input from appropriate sources used to inform the planning of the project? I 

Project staff engaged in several data collection activities in order to gain input from a variety of sources 
and there is evidence that project staff tapped into all of the key sources of input to inform the planning the 
project. 

Prior to the implementation of the project, staff had access to needs assessment data collected from the 
County Business Advisory Council. During the early stages of the project staff conducted interviews with 
foremen/supervisors to identify their general expectations and specific areas of need within specific job 
classes. A sampling of that data is listed below. 

Project Expectations As Described By Company Foremen/Supervisors 



Company 
Comdyne 



The Hall 
Company 



Grimes 
Aerospace 



LewiSystems 



General expectations 

• All workers need to 
know how to complete 
and read travelers that 
accompany each bottle 

• Expand basic literacy 
skills 

• Understand vocabulary 
used in industry 

• Carry out activities 
independently 

• Make fewer mistakes 

• Reading blueprints 

• Improving quality and 
productivity 



gain better opinion of 
self, self confidence 
improve basic skill levels 



Specific areas on need 

Inspectors: read blueprints 
Winders: improve attendance 
Valvers: read blueprints, math conversion 
Detailers: measure height, diameter, use scales 
and calipers 

Cutters: improve math, measurements read 
blueprints 

Assemblers: read and comprehend instructions, 

match numbers on blue prints 
Screen Printers: Skill related to SPC 



Repair Tec. basic reading, read ATP forms, reaa 

customer paperwork 
Analysts: improve math computation skills, 
CBT: Teaming skills, problem solving skills 
Model Makers: Math 

Assemblers: communication, listening skills 
Maintenance: higher skills 
Press Operators: basic reading, math, writing 
Molders: improve reading skills 
Secondary: reading spec, sheets to set job up. 



Project staff conducted on-site visits to participating companies to observe and shadow top workers within 
the specific job classes of individuals who would be involved in the training. A Literacy Task Analysis 
(LTA) was conducted on these positions to identify specific job related skills that should be incorporated 
in the training model. The summary of the LTAs were used to facilitate development of lessons and 
activities used in teaching the classes during Cycle I. 
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Project staff established a Problem Solving Committee and a Foremen/Supervisor work group which 
provided input and feedback to the process on an ongoing basis. 

The external evaluators collected additional information from participants and foremen/supervisors 
regarding their expectation for the training. The table below provides a sample of the most frequent 
responses that participants mado on selected question. 



Question 

What areas would you 
like to work on/ or 
learn? 



Do you think training 
help you do a better job 



Will classes improve 
your productivity 



Cycle I 
*Math 

♦Communications 
♦Writing 

Yes (30/55) 
no (9/55) 
Other (16/55) 

Yes (19/58) 
No(ll/58) 
Other (28/58) 



Cycle H 

♦Problem solving 
♦Getting along with 
others/team work 
♦Learn how others 
think and learn 
Yes (15/23) 
No (5/23) 
Don't know (3/23) 

Yes(8/23) 
No(6/23) . 
Don't Know(9/23) 



Cycle m 
♦Communication 
♦How to work with 
others 



Yes (16/29) 
no (4/29) 
Other (9/29) 

Yes (15/29) 
No (5/29) 
Other (9/29) 



Company foremen/supervisors were asked to provide feedback regarding their expectations of the project. 
Their most frequent responses to selected questions are presented in the table below. 



Question 



Resp 



onse 



What critical skills do the workers in your section 
of the company need to gain to better perform 
their tasks? 



How can the training contribute to increased 
productivity in your company? 

What changes in participant's behaviors do you 
expect to observe as a result of the training? 



Communicai : on skills 
Problem solving skills 
Team skills 

Understand instructions 

Reading 

SPC 

Cross-training skills 

Assess one's own contributions/self evaluation 
Collaboration/ team work methods 
Initiative( going beyond expectation) 
More communication among team members 



The participant and foremen/supervisor interview data was collected in conjunction with interviews with 
project staff conducted by the external evaluators. A Cycle I preliminary Evaluation Report was 
submitted to project administrative staff at the end of Cycle I. This report summarized the findings from 
participants, foremen, project staff and provided recommendation to facilitate planning for Cycles II and 
111. Many of the recommendations were implemented. 



Was adequate information available to develop good criteria used for selection of training program 
participants? 



i 



i 



Project staff obtained information from a variety of sources, and conducted several discussions with 
company representatives. It appears that, sufficient information was available to make appropriate decision 
regarding who should participate in the training. One issue described by project staff during interviews 
conducted during Cycle I was that even after several discussions with company representatives, there 
continued to be some lack of clarity regarding who should be involved in the training, what benefits they 
could expect to gain and how this information should be conveyed to potential participants. Also, the 
information communicated to company employees regarding the project was somewhat inconsistent across 
companies and the process that companies used to select participants was inconsistent. 
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The perception of key project staff was that companies had not given careful consideration to the impact of 
having a numbers of staff out of production for an extended period. In conjunction with the Cycle I 
Interim Evaluation Report the external evaluator made several recommendations including the following: 

• Provide a statement that companies can use for recruitment of volunteers or to select participants 
who will be involved in the training, 

• Outline criteria/skills that companies can use to tetter select potential participants, 

• Provide an outline of project goals, general classroom objectives to use with potential pa: ticipants, 

• Continue to provide an orientation session for new participants to clarify understanding of the 
project. 

| Were topics/activities pertinent to participant's job requirements identified? I 

Based on the data obtained from various sources including TALS tests, Learning Styles inventories, 
participant surveys, foremen surveys and interviews, and materials submitted to the external evaluator for 
review, the topics/activities were related to the job requirements of participants in a general way. 

During Cycle I the content proposed to meet the project goals were described as: 

Goal Content Training needs as perceived 

by participants and foremen 
Goal 1: Provide on-site workplace • 
literacy instruction to a minimum of 
300 employees of the four local 
businesses involved in quality control 
process. 

Goal 2: Provide basic knowledge and • Introduction and overview of • Basic literacy skills (e.g. 
skills necessary to participate in TQM Reading, math, writing skills) 

TQM/SPC processes, 

• Problem solving methodology • Measurement, blue print reading 
and tools 

• Team building and continuous 
improvement 

SPC: understanding graphs, 7 
SPC tools and charts 

• Gathering and interpreting data, 

Goal 3: Increase worker productivity • Metacognition • Improved self opinion 

leading to improved job maintenance, 

career advancement and decreased • Cognition: thinking skills, • Improved self confidence 

turnover, environmental factors, 

• Social factors 

• Problem solving, creative 
thinking, self confidence, stress 
reduction 

GOAL 4: Demonstrate a national 
rural workplace literacy model that can 
be replicated and to develop and 
disseminate a work-based curriculum. 



Based on interview data from project staff, it was determined that after Cycle I teachers felt they needed 
more information to understand the demands /tasks required of participants. Teachers wanted to spend 
time in the companies learning the culture and understanding various job roles, they wanted to develop a 
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better understanding of how the LTAs could be used a s a classroom tool. Teachers needed more time 
together to plan activities and share experiences in order to learn from each other. Observations and 
information obtained in staff interviews also indicate that as the project progressed, staff continued in staff 
development activities. Teachers were able to use information learned during Cycle I to improve prove 
lessons planned and implemented during Cycle II and improve upon Cycle II activities for Cycle DDL 



Were the project goals/objectives congruent with the goals and expectations of participants/ companies? 1 



i 



i 



I 
1 
i 
I 
I 



It is important to note here that during Cycle I, project staff determined that the goals as originally stated in 
the proposal should be revised/modified to convey a more focused representation of the scope of work that 
was doable by the project. Initially the project had elaborated seven(7) project goals that would be 
accomplished. The broad nature of the goals as stated appeared to be beyond the needs of the projects 
clients and beyond the scope of the feedback obtained from participants, the problem solving committee 
and project foremen/supervisors. The revised goals are stated in the table above. Two of the four(4), 
Goals 2 and 3 are most congruent with the goals and expectations stated by participants and foremen. In 
achieving Goal 2 project staff were required to teach skills that will enable participants to work more 
independently and more collaboratively within their work environment These characteristics in addition 
to content specific skills were viewed as valuable. 

Survey data obtained from company foremen during Cycle II and Cycle HI indicate additional skills that 
facilitate efficiency in the work environment and provide further evidence that these goals are congruent 
with the goals and expectations of the workplace. 



These include: 
GOAL 

GOAL 2: Provide basic knowledge 
and skills necessary to paiticipate in 
TQM/SPC processes, 



Content 



Skills observed 
foremen 



perceived by 



GOAL 3: Increase worker 
productivity leading to improved job 
maintenance, career advancement and 
decreased turnover, 



Introduction and overview of 
TQM 

Problem solving methodology 
and tools 

Team building and continuous 
improvement 

SPC: understanding graphs, & 
SPC tools and charts 

Gathering and interpreting data, 

Metacognition 

Cognition: thinking skills, 
environmental factors, 

Social factors 

Problem solving, creative 
thinking, self confidence, stress 
reduction 



Problem solving 

Taking the next step on their own 

Resolving problems 

independently 

Team camaraderie 

Less mediation by supervisors 



Process Phase 



jWere appropriate information inputs used in the management of the project? 



Project staff used a variety of feedback throughout the course of the project Many of these sources were 
described under the program Input Section of this report (see pg.4). Interviews with project staff indicated 
that teachers conducted a variety of activities in preparation for classes to aid them in planning including, a 
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variety of assessments (e.g.. Job Skills Inventory, Learning Styles Inventory, Lifetime Learning Log, etc. 
Other sources of feedback are described below. 

Participant Feedback 

Project staff was requested by the project evaluator to administer and collect the participant pre and post 
surveys during each of the three cycles. Participants responded to a number of questions related to their 
learning expectations, job skUls needed, and perceived benefits of the training. This information provided 
additional baseline information that could be incorporated into planning class sessions and post training 
responses provided feedback regarding the participants perceived satisfaction with the training and some 
suggestions for improvements. At the end of Cycle I the feedback from the participant surveys was 
summarized by the project evaluator in a preliminary report and provided to project managers. Project 
staff convened a lunch meeting for selected project participants. During the meeting several participants 
from four companies involved in the training showcased their learnings and provided a critique of the 
training to project managers and company supervisors. 

Feedback from Foremen/Supervisors 

Project evaluators conducted several structured interviews with company foremen/supervisors. The 
purpose of these interviews was to obtain feedback regarding their expectations, their perception of what 
participants were learning, how it was being applied to the work environment, and their satisfaction with 
the project The Project Director was invited to participate in these interviews and to utilize this 
information to guide changes to the project's activities. Ongoing meetings were conducted with the 
foremen/supervisor workgroup and the Problem Solving g Advisory Committee; These groups were very 
instrumental in providing guidance to project administrative staff. 

Feedback from Project Evaluators 

Project evaluators drafted a preliminary report upon completion of Cycle I. This preliminary report 
offered an evaluation summary of Cycle I activities from the perspective of project participants, 
foremen/supervisors, and project staff. It included a summary of standardized test results, and 
recommendations to project administrators regarding potential modifications that would strengthen project 
activities. 



Were management decisions effective? I 

There is limited data to support the effectiveness of management decisions beyond information obtained 
from project staff interviews, personal interactions and observations. However, evidence indicates that 
the management decisions were effective and benefited the project 

As mentioned above, an interim evaluation report was submitted to project administrative staff at the end 
of Cycle I which included several recommendation related to project planning and communications, needs 
assessment and curriculum development, project implementation and project management. Many of these 
recommendations were incorporated into the overall planning and implementation of the project. For 
instance, the participant selection process was clarified, the ABLE was discontinued and other more 
informal assessment measures were incorporated, time was allocated between Cycles I and II for project 
teachers spend time learning about the companies, ongoing training and planning opportunities were 
incorporated into the project, and the time equivalent for the project coordinator and teachers was increased 
to facilitate more opportunities for planning and communications. 

Project foremen/supervisors were asked in an interview to provide feedback on their perception of the 
effectiveness of project staff. The majority of the seven respondents provided positive comments about 
project staff. Project staff were described as having the following qualities: 

* Flexible and interested in knowing the company's environment and problems 

* Willing to make changes to meet the company's needs when necessary 

* Maintained good communication with companies 

* Management meetings provided good opportunity to exchange concerns/suggestions 
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Overall, the project staff appeared to have established and maintained a good rapport with their contacts 
within the participating companies. The one staff turnover which occurred near the end of the project 
appeared to have limited if any impact on the overall operations or implementation of the project. 

| Were training activities applied effectively? I 

Evidence to support the effectiveness of training activities indicates that the teachers were effective and that 
the training had a positive impact on participant's behavior. Data which supports this was gathered through 
interviews with teachers, training participants and foremen/supervisors. Project teachers participated in a 
structured interview and responded to questions related to class preparation and instructional implementation. 
The table below summarizes their responses to selected questions that reflect the processes used to prepare 
for and deliver instruction. 

SELECTED QUESTIONS FROM TEACHER INTERVIEWS (CYCLE III) 



Class Preparation: 



Question 


Teacher A 


Teacher B 


3. Conceptual 
preparation for 
class 

i 


Using outcomes or k term learning goals as 
a basis for teaching 

Allowing for flexibility in class structure to 
address topics that students suggest in addition 
to my predetermined goals 

Flexibility to teach basic skills needed to learn 
new concepts 

Determining when participants need more (or 
less) structure 


Determining how to present the problem to the 
student 

Generating ways to help students engage in 
problem solving 

• Keeping a log of activities that occur in prior 
classes to draw from in planning future class 
activities 

Determining when participants need more (or 
less) structure 


4. What teaching 
strategies or 
methods used 


Cooperative learning/education (teams monitor 
their learning) 

• Jigsaws 

S tuden t centered learning 

Providing practice opportunities 

Peer teaching/coaching ( using stronger 
students to help students experiencing 
di Iculty) 

Incorporating examples of work place behavior 
in teaching 

Providing focused application of skills to work 
environment 

Group problem solving 

Peer modeling and questioning 

Mediated learning (get students to think about 
and verbalize what they are doing how they do 
it and how to generalize it to other settings) 


Hands on activities 

Numerous practice opportunities 

Demonstration 

• Focused application of skil'o to work 
environment 

Incorporating examples of work place behavior 
in teaching 

Mediated learning (get students to think about 
and verbalize what they are doing, how they do 
it and how to generalize it to other settings) 
Cognitive coaching 

Problem solving 


5. Materials used 
for class 


Workplace materials (examples include blue 
prints, shop papers, routing materials, etc.) 
Teacher developed materials 
Commercial materials 


Workplace materials (examples include blue 
prints, shop papers, routing materials, etc.) 
• Teacher developed materials 
Commercial materials 
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INSTRUCTIONAL IMPLEMENTATION 



Question 


Teacher A 


Teacher B 


6. Content focus 


• Learning styles profiles and learning style 
inventory 

• Jigsaws 

Math and math embedded in language 

• TQM concepts 

• Large group discussion of day's .activities 


• Metacognative process and it's application to the 
workplace environment 

• Communication skills 

• Problem solving 

• Basic skills application (reading/math) 
Location specific topics include: 

• Handling job stress 

• Managing work loads and reduced cycle times 

• Using work related problems/projects to for skill 
application 


7. Teaching 
TOM/SPC 


• In some not all locations 


• In some not all locations 


8. Sequence of 
instruction 


• Learning styles concepts are presented early, 

• Practice of concepts until it become a habit. 

• Thinking styles, thinking about 
communication and thinking strategies. 

• Activities that include reading ma'ii, basic 
skills, cooperative learning, team library, 
jigsaws 

• Class presentation on some concept learned in 
class. 


• Make a determination of overall class needs 

• Assess where class is (readiness level, learning 
styles, etc.) 

• Obtain input from the class regarding what they 
want/need to learn 


9. Organization 
(lesson plan 
format/ 

individualizatio 
n/group focus) 


• Felt that lesson plans were limiting 

• No concrete way to determine if objectives are 
met. 


• Conduct assessments 

• Identify learning needs 

• Determine learning go;, 5 

• Teach p rocess approaches (e.g Metacongnitive 
approach, cognitive coaching, learning styles, 
problem solving) 

• Integrate process with content through exercises, 
practice, team activities, work place 
projects/activities, etc. 


10 Class 
framework 


• Application 

• Concepts 

• Problem solving experience 

• Problem solving, learning styles and 
communication are emphasized throughout all 
activities. 

• The rate of presentation will vary from class to 
class based on their needs. 


• Metacognnition 

• Transition and practice 

• Job application(continuous improvement or job 
revision) 

• Discussion of what was learned 

• Feedback from class on how to improve ae 
activity. 


11. Percent time 
on major 
concepts: 
(general 
approximation) 


• Problem solving most 

• Learning styles most 

• Communication skills written some 

• Communication skills verbal some 

• Basic skills (math/reading) some 


• Application most 

• Practice most 

• Process most 

• Demonstrating some 


14. How do you 
measure 
outcomes? 


• Observation of class behavior, teamwork, 
cooperative/learning 

• Anecdotal comments from participants about 
how they apply what they learn to situations 
outside of the learning environment. 


Job skills inventory (pre and post) 

• Anecdotal comments from participants that 
demonstrate how they use skills learned in class in 
other situations. 

• Use problem situations from the work 
environment as cases in class. 
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Additionally, teachers met regularly to plan and discuss activities carried out in their respective classes, 
they observed and provided feedback to each other and received feedback from the project director and 
project consultant 

Feedback from project foremen/supervisor interviews further supports their perception of the effectiveness 
of training activities. A summary of foremen/supervisor responses to selected questions is provided 
below: 
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What Critical Skills do the workers in your section need to better perform their tasks? (pre training survey) 

• problem solving 

• design 

• team skills 

• quality, 

• programming, etc. * 

Your sense of what participants are learning? (pest training) 

• participants learned how others think 

• how to communicate with others differently, 

• participants learn that they have an obligation to the listener. 

• participants learn to take the risk to speak out 

• participants show respect for others 

• learned that another's opinion is worthwhile 

• participants learned that individuals learn differently, 

What skills have you seen participants apply?(post training) 

• problem solving, taking the next step on their own, 

• team camaraderie 

• increased comprehension 

• increased confidence has increased their ability to take the initiative. 

• participants require less mediation by supervisors, 

• participant's minutes has changed 

The perceptions of foremen/supervisors support that the training was effectively implemented. 
Outcome Phase 

| Were the goals and objectives of the project achieved? 1 

Goal 1: Provide an on-site workplace literacy instruction to a minimum of 300 
employees of the four local business involved in quality control process 

The four companies located in the Champaign County that participated in the project are: LewiSystems, 
Grimes Aerospace, The Hall Company and Comdyne. Based on data requested from project staff the 
following table shows the number of employees that participated in the workplace literacy instruction 
project according to site (Grimes Aerospace had participants at separate instructional classes at different 
sites) and training cycle. 



Company/Site 


Number of Employees Trained 


Cycle I 


Cycle II 


Cycle 
III 


Total 


Comdyne 


11 


10 


19 


40 


Grimes - Rt. 55 - Operations 


22 






22 


Grimes - Rt. 55 - CBT 


10 


11 


8 


29 


Grimes - Twain Ave. - Prod. Supt. 


14 


15 




29 


Grimes - Russell Street 




22 


39 


61 


Hall Company 


19 


18 


16 


53 


LewiSystems* 


8 


8 


5 


21 


Total 


84 


84 


87 


255 



* Participants at LewiSystems during Cycles I and II were the same. Classes extended over two cycles with one-half of the 

hourly load per week. 
Note: data presented on this table is based on figurrs received from project staff. 
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The project goal was to reach a minimum of 300 participants. The instruction reached 255 participants. 
The instruction did not reach the goal number due to unexpected circumstances which were outside of the 
project's control. Such external factors were related to the local companies' restructuring and downsizing. 
During the project period, several participants were laid off from all company sites. This restructuring of 
the companies affected the availability of employees that could participate in the training from *.ome of the 
smaller companies, and/or reduced the amount of employees willing tc participate in the training from 
several sites. < 

Project staff utilized proactive strategies to disseminate information about training classes both within the 
companies and to audiences external to the project We can conclude that one of the main reasons to 
which the project was short of reaching the established goal for number of participants can be attributed to 
circumstances that were beyond the project's control. 

Goal 2: Provide basic knowledge and skills necessary to participate in TQM/SPC 
processes. 

Goal 3: Increase worker productivity leading to improved job maintenance, career 
advancement and decreased turnover (skills that lead to increased worker productivity 
as laid out by the workplace literacy project) 

Goals 2 and 3 were analyzed together because several overlapping skills, which are identified in both 
Goals, were evaluated by the same procedure by project evaluators. Goal 3's identified skills that lead to 
increased productivity are measured by the same tests and interview/surveys utilized by the analysis of 
Goal 2. The measurement of productivity gain and job maintenance will be analyzed separately. 

To meet the project goal of involving workers in TQM/SPC processes (Goal 2), the project staff planned 
to instruct the following items: 

♦ understanding of graphs and charts, 

♦ interpreting data, 

♦ creating charts, 

♦ application of data to problem solving, 

♦ units of measure, 

♦ SPC tools, 

♦ instruction and overview of TQM, 

♦ problem solving methodology, 

♦ problem solving tools, 

♦ team building, 

♦ continuous improvement. 

To meet the project goal of increasing workers' productivity (Goal 3), the project team designed the 
following instruction items: 

♦ metacognition (including oral language, written language, mathematics, math embedded in 
language, reading), 

♦ cognition/metacognition (including thinking skills, environmental factors, social factors, problem 
solving, creative Slinking, self-confidence, stress reduction). 

To assess this outcome's achievements, evaluators utilized standardized tests (ABLE, TALS) and pre- and 
post- interviews and surveys with foremen/supervisors of the training participants. Workers development 
of items such as written language, mathematics, math embedded in language, reading, interpreting data, 
application of data to problem solving and units of measure was assessed by the analysis of standardized 
pre- and post-tests. All other items were analyzed through perceptions and qualitative assessments 
obtained through interviews and pre- and post-training surveys completed by foremen and supervisors 
which were collected by the evaluators throughout the nroject. 
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Standardized Tests 

The pre- and post- analysis of the standardized tests was performed by using two basic methods: 
interpreting the scores obtained by the students according to a manual (TALS Administration and Scoring 
Manual) and by statistical analysis of the tests' raw data (t-test of paired means and percentages). For 
more details on the methods of analysis, please refer to Section 3 - External Evaluation Methodology. 

The three components of the TALS test (prose, document, and quantitative) were utilized to measure a 
broad profile of the basic skills of the participants. The prose test related to the Goal 2 of the project by 
providing a measure of reading and written language skills. Prose tested the students' knowledge and 
skills needed to understand and use information contained in various kinds of textual material (Education 
Testing Service, 1991). Three skills were tested: locating information in text, and integrating and 
generating information from text. The document component of the test was used to measure workers' 
skills in processing information from documents. More specifically, this test was used to measure the 
workers literacy skills to locate and use information contained in materials such as tables, schedules, 
charts, graphs, maps and forms, which are part of the instruction items on Goal 3. The quantitative 
component of the test provided a measure of participants skills to perform basic mathematics operations 
and use numbers embedded in printed materials (skills required for Goals 2 and 3). 



Components of the Tests of Applied Literacy Skills (TALs) 



Prose 


Document 


Quantitative 


Understand and use information 
contained in various textual 
materials. 


Process information found in 
documents. 


Complete quantitative tasks such 
as arithmetic operations and use 
numbers embedded in printed 
materials. 


Skills identified in the project 
Goal 3: 

♦ reading; 

♦ written language. 


Skills identified in the project Goal 
2: 

♦ understanding graphs and 
charts; 

♦ gathering data; 

♦ interpreting data; 

♦ units of measure. 


Skills identified in the project 
Goals 2 and 3: 

♦ mathematics; 

♦ math embedded in language. 


Interpreting scores (TALS): 

To use as a reference for participants' skills development, we compared their results of pre- and post- tests 
to the national difficulty level score as an identifier of difficulty of general tasks for each test component 

Average National Difficulty Levels 


Prose 


Document 


Quantitative 


Locating information in text: 
293 


Locating information in documents: 
242 


Addition tasks: 235 
Subtraction tasks: 298 


Integrating information from 
text: 325 


Cycling through information in 
documents: 308 


Multiplication: 318 
Division tasks: 326 


Generating information from 
text: 372 


Integrating information in 
documents: 318 


Combination tasks: 363 



1 



It is important to note that the national difficulty levels serves in this study exclusively to provide a 
reference for analysis of what tasks the participants likely improved from the training. The pre- and post- 
test statistical analysis of the raw data (next subsection) provided information which complements the test 
score analysis. 
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An analysis by test component follows. 



Prose Component: 

On Cycle I, Comdyne, Grimes - CBT, and LewiSystems moved to a higher proficiency level (Table 1). 
Grimes CBT was the only class to surpass the national average at the most difficult task (generate 
information from text). On Cycle II, Comdyne, Grimes Twain Ave., and Grimes - Russell Street moved 
to a higher proficiency level by surpassing the national difficulty level on integrating information from 
text On Cycle DDE, the aggregate of companies moved to a higher proficiency level. All classes 
experienced an increase in their mean scores, with the highest growth experienced by the class at 
Comdyne. 

Document: 

The document component has the lowest national difficulty level of the three components. And all classes 
presented a pre-test proficiency level on the most difficult task: integrating information in documents 
(Table 2). However, the average results for all companies was low and did not improve significantly on 
any of the three cycles. 

Quantitative: 

On Cycle I quantitative skills were measured by using the ABLE test The results of the ABLE test will be 
presented and discussed in the statistical analysis of the raw data. On Cycles II and m, the document 
component of the TALS test was adopted by the project team to measure quantitatu 2 skills of participants. 

The quantitative component of the TALS test has several proficiency levels (addition, subtraction, 
multiplication, division, and combination tasks). Grimes CBT class on Cycle I surpassed the national 
difficulty level on all tasks (Table 3). Comdyne (classes of cycles II and HI), Grimes Russell Street a.m. 
and p.m. classes (cycle HI) stayed at an intermediate proficiency level equivalent to the average 
multiplication and division national difficulty level. The Hall company experienced notable progress in 
this area on Cycles II and in. 

Statistical Analysis of the Raw Data 

The analysis of the raw data does not provide an estimate of which specific concepts were mastered by the 
students between the time they took the pre- and the post- test. However, it provides an indication of the 
change that occurred during the period in the number of absolute correct responses and its significance 
according to a pre-determined confidence level (please refer to Chapter 3 - Evaluation Methods for more 
detailed information). 

Prose: 

The aggregate result of all participants demonstrates a statistically significant growth on all cycles (Table 
4). In other words, students had, in average, scored more points on the post-test when compared to the 
pre-test results for all cycles on the prose component. However, individual classes may have experienced 
a reduction or stayed at a same level of response level between pre- and post-tests. Cycle HI is where the 
growth was more noticeable. 

Document: 

As it was discussed on the "Score" analysis, the document component was the item in which the aggregate 
of all participants did not experience growth (Table 5). In reality, on Cycle in, the aggregate of all classes 
shows a statistically significant decrease on the average score (raw data) from pre- to post-test 

Quantitative; . 

On Cycles I and II, the aggregate of all classes experienced a statistically significant mcrease in scores 
(raw data) between the pre- and post-tests (Table 6). On Cycle HI, results are not conclusive for all 
classes. On Cycle I, Comdyne, Grimes - Twain Ave. and the Hall Company experienced significant 
increase in scores (raw data). On Cycle II, Comdyne, Grimes, the Hall Company and specially Grimes - 
CBT experienced statistically significant increases in scores. Grimes - Twain Ave. (Cycle II) was the only 
class to experience a reduction on the score from pre- to post-test. 
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Relationship between TAL S' Results and Pro ject Goals 2 and 3 

As discussed earlier, the standardized tests measured some of the instructional skills present in Goals 2 
and 3 of the Project Reading and written language are part of Goal 3 (Increase worker productivity 
leading to improved job maintenance, career advancement and decreased turnover). According to the 
TALS results, reading was one of the skills in which growth was experienced across all project cycles for 
the aggregate data (data aggregated for all classes during each cycle). In terms of proficiency levels, six 
classes achieved results that placed them on higher national difficulty levels. Participants reading skills 
were positively affected by their participation on this training. 

Results on quantitative skills, which are skills included in Goals 2 and 3, showed general growth on 
Cycles I and II. Results generated for Cycle HI are inconclusive. 

The standardized test results on document skills, related to PC processes (Goal 2), indicate the participants 
did not experience significant growth as a result in participating in the project 
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Interviews and Surveys 



The interviews and surveys administered throughout the project provided information regarding several 
instructional items not covered by the standardized tests. Such items are: in Goal 2, introduction and 
overview of TQM, problem solving methodology, problem solving tools, team building, continuous 
improvement; in Goal 3, cognition and metacognition (thinking skills, environmental factors, social 
factors), problem solving, creative thinking, self-confidence, stress reduction. 

Surveys and Interviews with Foremen/Supervisors 

From surveys and interviews with foremen performed at the end of Cycle n, we obtained the following 
feedback regarding observed skills participants obtained from training: 



Grimes Twain Avenue 


Observed Improved Skills/Behaviors 
(from foremen interviews/surveys) 


Relationship with Project Goals 2 and 3 


Communication skills: "More interaction on work 
floor. Learning how everybody else thinks help workers 
understand each other. Supervisors need to interfere less 
in the workers interactions." 


Goal 3 : Metacognition (oral language) 

Cognition (environmental factors, social factors) 

Self-confidence 

Stress reduction 


Knowledge of the overall: foremen/supervisor 
indicated that in general the training provided workers 
with an improvement on the knowledge they have of the 
overall process. 


Goal 2: Team building and cjntinuous improvement. 


Problem solving: "...for example at the reception area, 
they try to solve problems themselves. Workers that 
have been involved in the training take the initiative to 
solve problems." 


Goal 3: Problem solving 
Goal 2: Problem solving methodology 
Problem solving tools. 


Teamwork: foremen/supervisor indicated that in general 
workers acquired better team work skills as a result of the 
training. 


Goal 2: Team building 


Reading: foremen/supervisor indicated that training 
nrovided workers with better reading skills. 


Goal 3: Metacognition (learning how to learn, including 
"reading") 


The Hall Company 


Observed Improved Skills/Behaviors 
(from foremen interviews/surveys) 


Relationship with Project Goals 2 and 3 


Communication skills: "The training has helped by 
breaking down department barriers and allowing 
employees to be at easy with communicating with 
others." 


Goal 3: Metacognition (oral language) 

Cognition (environmental factors, social factors) 

Self-confidence 

Stress reduction 


Problem solving: "Better problem solving by the 
individual worker who now feels that they can make 
decisions on their own." 


Goal 3: Problem solving 
Goal 2: Problem solving tools 

Problem solving methodologies 


Teamwork: foremen/supervisor indicated that in general 
workers acquired better team work skills as a result of the 
training. 


Goal 2: Team building 


Basic skills on other jobs: foremen/supervisor 
indicated that in general the training provided workers 
with an improvement on the knowledge they have of the 
overall process. 


Goal 2: Team building and continuous improvement 
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Conidyne 



Observed Improved Skills/Behaviors 
(from foremen interviews/surveys) 


Relationship with Project Goals 


Communication skills: "Good communication 
(between workers and supervisors). Project staff 
participated in management meetings and 
concerns/suggestions were exchanged." 
"Greater acceptance of other's jobs/limitations, ask others 
for help (between workers)." 


Goal 3: Metacognition (oral language) 

Cognition (environmental factors, social factors) 

Self-confidence 

Stress reduction 


Problem solving: "Workers solve problems 
themselves. Less problems to be mediated by 
management level." 


Goal 3: Problem solving 

Problem solving methodologies 


Teamwork: "Improvement in setup for next production 
line. Previously workers would walk away. Now they 
show interest" 


Goal 2: Team building 



LewiSystems 


Observed Improved Skills/Behaviors 
(from foremen interviews/surveys) 


Relationship with Project Goals 


Communication skills: "Some participants improved 
their communication skills." / 


Goal 3: Metacognition (oral language) 

Cognition (environmental factors, social factors) 

Self-confidence 

Stress reduction 


Problem solving: "Some participants improved their 
problem solving skills " 


Goal 3: Problem solving 
Goal 2: Problem solving tools 

Problem solving methodologies 


Team work: "Some participants improved their 
teamwork skills." 


Goal 2: Team building 


Knowledge of the overall process: "Some 
participants gained knowledge of the overall process." 


Goal 2: Team building and continuos improvement 


Basic literacy: "All participants increased their basic 
literacy skills." 


Goal 3: Metacognition (learning how to learn, including 
"reading") 

Goals 2 & 3: Mathematics, math embedded in language. 


Grimes - Russell Street 


Observed Improved Skills/Behaviors 
(from foremen interviews/surveys) 


Relationship with Project Goals 


Communication skills: "Better communication. Now 
people "listen and hear." People seek clarification. 
There is better understanding of each other. Keeps people 
from going on wrong directions on task, or doing it 
twice because they were not listening well before." 


Goal 3: Metacognition (oral language) 

Cognition (environmental factors, social factors) 

Self-confidence 

Stress reduction 


Stress management skills: "Lay-offs set people back 
here. Stress management skills were very helpful to 
cope with lay-offs. The project was very good to help us 
through this tough phase." 


Goal 3: Stress reduction 


Problem solving skills: "Workers developed better 
ways of doing same tasks. They are thinking "outside of 
the box", finding alternative ways of doing tasks t smart 
ways, learning from each other." "Require less problems 
to be mediated by management level staff." 


Goal 3: Problem solving 
Goal 2: Problem solving tools 

Problem solving methodologies 
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Post-training Surveys from Training Participants 

At the end of the training, participants were asked whether they had a better understanding of how to use 
Statistical Process Controls (SPC) and Total Quality Management (TQM) on their jobs. The response 
frequencies are depicted below: 

Do you have a better understanding of how to use SPC on your job? 
Cycle I 

Yes No Other 

Comdyne 2 
Grimes-Twain 7 
Hall Company 11 



6 
3 
1 



0 
0 
1 



Total 



20 



10 



Cycle III 
Comdyne 



Yes 



No 



Other 



0 



Do you have a better understanding of how to use TQM on your job? 
Cycle I 

Yes N o Other 

Comdyne 
Grimes-Twain 
Hall Company 



3 
4 
11 



4 
5 
1 



0 
0 
1 



Total 



18 



10 



Cycle III 
Comdyne 



Yes 



No 



Other 



0 



i 



I 



:RIC 



In general, Cycle I and Cycle HI respondents indicated that they had a better understanding of how to use 
SPC and TQM on their jobs. Comdyne being the only exception on Cycle I. On Cycle HI, Comdyne 
participants indicated that they had a better understanding of huw to use SPC on their job. 

To conclude, from the standardized tests, which cover literacy, document, and quantitative general areas 
of knowledge, the overall results indicate that all training classes remained at their original proficiency 
level or moved to a higher proficiency level during the period of time between the dates in which the pre- 
and the post-tests were submitted (beginning and end of training respectively). 

From the interview and survey results, we conclude that the project's main accomplishments are in the 
areas of improving participants communication, problem solving and team work skills. These 
instructional items are components of Goals 2 and 3 of the project 

Goal 4 - Demonstrate a national rural workplace literacy model that can be replicated 
and to develop and disseminate a work-based curriculum. 

The project evaluators received only a draft outline and draft copy of the Workplace Literacy Curriculum. 
Based on draft information received to date, the curriculum being developed is based on the principles the 
project utilized during the training period. The principles were developed based on the project's 
assessment that "global economic concerns demand employees who demonstrate a high level of 
competence to deal with complexity and change." 1 

It is the project's belief that to answer to such demands, the traditional teaching methods have to be 
changed. The project's answer to this question is a curriculum model which assimilates the complex 



1 From Workplace Literacy Curriculum - Explanation, Development, and Thematic Issues/Usson Plan - draft 
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workplace environment and utilizes cognitive thinking and student-directed learning with cooperative 
education methods (please refer to Appendix E for a summary of the curriculum model as proposed by 
project staff). 

1 Did the""training impact the workers opportunities for advancement? ] 

During the period covered by the three training cycles, the participant companies experienced restructuring 
and downsizing of their organizations. This reorganization during the training period, translated into 
several employees being laid off from all company sites. Company downsizing negatively affected 
employees' opportunities for advancement. Since company restructuring is external to training and may 
overshadow the training impact, the opportunities for advancements were not evaluated. 

Did the training / intervention meet the workers' skill development 
needs? 



The answer to this evaluation question can be obtained by comparing the responses obtained on item six of 
the pre-training participant survey (What educational areas would you like to work on?) with the responses 
obtained on item four of the participants' post-training surveys (Were the educational areas that you 
wanted to work on addressed in the class?). A problem posed to this analysis is that not all sites 
completed the post-training surveys. Project evaluators received 35 completed surveys on Cycle I (from 
three sites), none on Cycle II, and 10 from Cycle III (one site only). The respondents response frequency 
is presented in the table next page. 



Cycle I 

Needs Assessment input from project participants obtained from pre-training survey on 
Cycle I. "What job skills do you need to work on?" 



Skills 


Hall 


Grimes* 


Corndyne 


Total 


Technical skills related to my job 


4 


10 


1 


15 


Computer skills 


1 


3 


1 


4 


Knowledge of overall process 


3 




4 


Communication skills 




3 




3 


Math 


3 






3 


Basic skills on other jobs 




1 


1 


2 


Reading 


2 


1 




2 


Writing 


1 




2 


Other 




2 




2 


Total 


14 


20 


3 


37 



* Grimes column includes three sites: Rt55 Operations, CBT, and Twain Ave. 
Note: Results tabulated from open-ended type of question. 



Cycle I 

Were the educational areas that you wanted to work on addressed in the class? 



Yes No Other 

Comdyne 4 3 1 

Grimes-Twain 2 5 1 

Hall Company 4 6 2 

Total 10 14 4 



Most respondents of Cycle I indicated that the areas in which they received training were concentrated in 
the areas of communication skills, learning styles, and problem solving skills. Most of the respondents 
indicated that they wanted the training to cover more job related subject areas, which is compatible to what 
they identified as areas of need at the beginning of the training. 
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On Cycle EI, participants identified the three most important areas of need as being teamwork skills, 
communication skills, and problem solving. The post-training data available is not significant (one site 
only) to confirm the project's accomplishments in this area. However, considering that the project did not 
change its basic curriculum, it is admissible to assume that participants from Cycle EI had the educational 
areas that they wanted to work on addressed in the training. 

Cycle III 

Needs Assessment input from project participants obtained from pre-training survey on 
Cycle III: "Please check all the areas you wc id like to work on this class." 

Skills Grimes Lewis 

Comdyne Russell Systems Total 



St. 

Teamwork skills 12 4 6 22 

Communication skills 10 6 6 22 

Problem solving skills 10 5 5 20 

Knowledge of overall process 6 5 5 16 

Total Quality ; Management (TQM) 6 4 5 15 

Technical skills related to my job 6 4 5 15 

Basic skills on other jobs 4 4 4 12 

Statistical process controls (SPC) 3 3 3 9 

Writing 13 3 7 

Reading 2 12 5 

Basic literacy 1 0 4 5 

Total 61 39 48 148 



Note: Results tabulated from multiple choice type of question. 



Cycle III 

Were the educational areas that you wanted to work on addressed in the class? 

Yes No Other 

Comdyne 2 2 3 



1 What was the workers' perceived satisfaction with their involvement in the training? 

Items one (Do you think your company should provide classes for courses like this) and seven (were your 
overall expectations for this class met?) are utilized to assess the participants' level of satisfaction with 
their involvement in the training. On Cycle I, most respondents thought their companies should offer 
classes for courses like this. Most of the negative responses and some of the positive responses indicated 
were justified on the basis that the respondents would like classes more dedicated to job related skills. 

Do you think your company should provide classes for courses like this? 



Cycle I 

Yes No Other 

Comdyne 6 2 1 

Grimes-Twain 8 4 0 

Hall Company 9 4 1 

Total 23 10 2 



Cycle III 

Yes No Other 

Comdyne 6 2 0 



Most participants that responded to the Cycle I post-training survey indicated that the training did not meet 
their expectations. Many respondents indicated that they were not able to respond to this question because 
they did not have much knowledge of what would be covered in the classes. 
The majority of the Comdyne participants of Cycle III who responded to the survey indicated that the 
training met their overall expectations. 
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Were your overall expectations for this class met? 
Cycle I 

Yes No Other 

Comdyne 2 4 2 

Grimes-Twain 2 7 3 

Hall Company 2 9 2 



Total 



20 



Cycle III 
Comdyne 



Yes 



No 
0 



Other 



Training Impact 

What was the foremen/supervisors' perceived satisfaction with the 
training? 



To what extent did the intervention impact the worker's levels of productivity? 

Aside from the analysis of the instructional components that the training anticipated as leading to increases 
in productivity, the e valuators designed a post-training survey for the participants and a pre- post-analysis 
of productivity data - field information - for each participating company (Appendix D). The data forms 
were sent to the supervisors/foremen for each of the sites. These data forms were not completed and 
mailed to the project evaluators on the specified deadline. The evaluators sent a follow up letter with more 
copies of the materials, extending the deadlines. Only one response was obtained from the project sites 
which could not be representative of the entire group. 

Since participant companies did not provide the requested information, project evaluators interviewed 
foremen/supervisors at the completion of Cycle II and after the completion of Cycle HI. Another source of 
information to answer this evaluation question are the pre- and post-training surveys to participants, on 
Cycles I and HI. 

Foremen/Supervisors Interview 

The following is a summary of the foremen and supervisors responses regarding productivity gains from 
training (Cycles II and III). 

Grimes CBT: Lay offs that occurred during the period of the training interfered with any effects the 
instruction may have had on participants. As reported by the CBT supervisor when asked about 
productivity changes before and after the training period: "People feel helpless. There isn't enough people 
to do the work at present time after all lay offs." 

Grimes - Twain Ave.: "We didn't expect any [increase in productivity]. Goals in the program were 
not to come from individual effort but from process improvement. Maybe in the long term, as team effort 
helps improve the process, we will experience increases in productivity. Self confidence has improved, 
what helps to improve the process, as well as increase in communication across categories (e.g. design 
people obtaining input from assembler). 

The Hall Company: According to the supervisor interviewed by the project evaluation team, the 
project contributed positively to increase productivity. The supervisor indicated that the training helped 
increase productivity because it generated: 

• Increase in "general knowledge and understanding of own contribution and contribution of 
others;" 

♦ "Better understanding of total picture, [employees] view job as part of overall function;" 
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• "Increased appreciation;" 

• "Reduction in re-work." 

The indicators of increased productivity pointed by the supervisor were: historical quality, increased team 
work, increased interactions (reduction of barriers), number of jobs done within estimated planned hours, 
scrap reduction, demonstrated understanding of company goals, and the use of English in customer 
reports. x 

LewiSystems: Parallel to the training period, LewiSystems experienced a re-design of its production 
lay-out The supervisor indicated to the project evaluation team that such restructuring interfered directly 
with the company's overall productivity levels. Therefore, any productivity gains or losses gathered from 
pre- and post-training productivity data would not be necessarily an implication from the training. Based 
on this rational, the supervisor declined to comment on productivity gains or losses experienced by the 
participant workers during the training period. 

Comdyne: "According to charting based on productivity and scheduled shipment, we observe a slow 
upward trend in productivity with respect to time. Overall, there is much better communication between 
job areas. Management is obtaining good ideas from introverted workers. As workers solve problems 
themselves, there are less problems to be mediated by managers." 

Participants Post-training Surveys 

The participants post-training surveys included two questions regarding productivity: (a) Do you think 
this class helped you do your job better?, and (b) Do you think this class helped improve production levels 
in your company? The project evaluators received post-training surveys from Cycles I and HI only. From 
Cycle I, three sites only (Grimes Twain Ave, The Hall Company, and Comdyne) and from Cycle in, 
surveys from Comdyne only. Results show that participants do not think the training positively impacted 
companies' productivity. Responses were undecided with regard to whether training helped participants 
do their jobs better (please refer to the table below). 

Cycle I: Do you think this class helped you do your job better? 

Yes No 

Comdyne 2 6 

Grimes-Twain 7 4 

Hall Company 8 6 

Total 17 16 

Cycle I: Do you think this class helped improve production levels in your company? 

Yes No Other 

Comdyne 0 6 2 

Grimes-Twain 2 7 

Hall Company 5 6 1 

Total 7 19 3 

On Cycle m, only Comdyne results are available: 

Do you think this class helped you do your job better? Seven participants responded yes, 
four participants responded no. 

Do you think this class helped improve production levels in your company? Three 
participants responded^, four participants responded no. 



Cliampaign County Workplace Literacy Project Final Evaluation Report 39 



ERIC 48 



I 



I 



I 

i 
a 



er|c 



Summary 
Cycle I 

On Cycle I, training participants (Grimes - Twain Ave, Comdyne, arid the Hall Company) identified mostly 
technical skills related to their jobs as areas they would like to work on during the training. The training, 
however, provided more emphasis on subjects such as communication skills, team work, and problem solving. 
Most of the respondents indicated that their companies should provide courses like this as long as the courses 
would place more emphasis in the technical skills required for their job performance. 

Participants at Grimes (Twain Ave.) and the Hall Company, at the end of Cycle I, indicated that they had a 
better understanding of how to use SPC on their jobs after the training. Participants at Comdyne, during the 
same Cycle, indicated that the training did not provide them with better understanding of SPC on their jobs. In 
terms of understanding TQM, only participants from the Hall Company indicated benefits received from the 
training on Cycle I. 

The standardized tests indicated that, overall, participants improved their understanding and ability to use 
information contained in various textual materials (TALs, Prose) and their quantitative skills (ABLE). The 
ability of the participants to process information found on documents was not significantly affected by the 
training. 

When asked if they were satisfied with the training, most participants (Grimes at Twain Ave, Comdyne, and 
the Hall Company) indicated that the classes did not meet their expectations. However, they explained that they 
did not know what to expect from the training. Participants mentioned that they would like to have been better 
informed of what materials this training would cover, and what they would learn from the training. Some 
participants mentioned that participation in such type of training should be voluntary, instead of allowing 
supervisors to select who should participate. 

Overall, the participants (Grimes at Twain Ave., Comdyne, and the Hall Company) were undecided in the 
classes helped them do their job better. Most of them, at the end of Cycle I, thought that classes did not help 
improve production levels in their companies. 

Cycle II 

Information from Cycle II is very limited since participants did not return the post-training surveys. The 
analysis will be restricted to the standardized tests. On Cycle n, the average participant experienced a similar 
improvement as that of participants of Cycle I: the standardized tests indicated that, overall, participants 
improved their understanding and ability to use information contained in various textual materials (TALs, 
Prose) and their quantitative skills (ABLE). The ability of the participants to process information found on 
documents was not significantly affected by the training. 

Cycle III 

Cycle IB includes standardized test scores, Comdyne' s participants' opinion, and overall perceptions of the 
training by participants supervisors. 

The analysis of the standardized tests indicate that on Cycle III, the aggregate of all companies moved to a 
higher proficiency level in the prose section of the TALs test (their understanding and ability to use information 
contained in various textual materials). Despite the aggregate of the participants experience growth on all three 
training cycles, on Cycle III this impact was more noticeable. The aggregate results on the other parts of the 
test did not show participants' improvements or were inconclusive. 

Comdyne participants indicated that at the end of Cycle HI they had a better understanding of how to use SPC 
on their jobs. Most of these participants indicated that they did not achieve a better understanding of how to 
use TQM in their jobs d' : .ng the same period. 

Most Cycle HI participants indicated that the areas they would like to work on during the training were: 
teamwork skills, communication skills, and problem solving skills. We can observe here a difference from the 
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participants' identified areas on Cycle I, where participants indicated they would like to work in skills durectly 
related to their jobs. Two assumptions can be made here: 1) participants from Cycles I and II presented to 
their colleagues the materials they had been learning in training; and 2) participants from Cycle ffl knew what 
materials were covered during training and chose to participate in the. training. 

Su^^tlltdm7rZt"i\ss had different opinions of what were the training impacts on their companies. The 
LrSor at Grimes (CBT), indicated that downsizing and subsequent lay off of employees, during the period 
covered by the training, interfered with any effects the training may have had on the participants. 

At Grimes (Twain Ave.), the supervisor indicated that they were not expecting improvements in productivity in 
me short term. This supervisor indicated that in the long term, the training impacts of emp oyee increased self- 
confidence^d better communication across categories will increase team effort which could lead to improve 
the company's process, leading to increases in productivity. 

The supervisor at the Hall Company indicated that the project contributed positively to increases in productivity 
bv increasing the employees' general knowledge and understanding of the company and the process, by 
increasing their understanding of own and others' contribution to the process, by increasing employee 
appreciation of their work, and by helping reduce barriers across categories (improving employee s teamwork 
skills). The supervisor indicated the following indicators of increased productivity: historical quality, scrap 
reduction, use of English in customer reports. 

At the LewiSystems, the supervisor indicated to us that during the training period the company experienced a 
re-design of its production lay-out which interfered in any impact that could be obtained from the training. 

The supervisor at Comdyne observed a slow upward trend in productivity with respect to time. These 
results were obtained according to charting based on productivity and scheduled shipment 
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Conclusive Remarks 



Based on the results of the evaluation, the following conclusions regarding the accomplishments of this project 
are offered: 

1) Project staff obtained reliable sources of input and feedback to guide the implementation of the project, 
to make effective decisions regarding changes/modifications needed. 

2) Except for the projected number of participants expected to be served, Goals 1 and 2 were implemented 
within the context of needs expressed by clients served by the project (project participants and 
foremen/supervisors) 

3) Due to the numerous factors occurring within individual companies that were beyond the influence and 
control of this project, and the limited amount of feedback received from clients, it is not possible to 
conclude that the training contributed to a significant increase productivity leading to improved job 
maintenance, career advancement and decreased turnover or that Goal 3 was accomplished. 

4 . Project staff implemented a well designed model of instruction that can be used to facilitate learning, 
facilitate improved communication, problem solving and team building. 

The curriculum model being developed by the project staff is based on the principles the project utilized during 
the training period. The curriculum model is expected to assimilate the complex workplace environment and 
utilize cognitive thinking and student-directed learning with cooperative education methods. The training also 
uses facilitated process approach to integrating knowledge and information rather that using rote teaching 
methods to teach skills in reading and mathematics. For most participants and supervisors, this translates into a 
training activity that helps participants improve communication, problem solving, and teamwork skills. This 
approach appears to be beneficial in cases where relatively small companies, or industrial sites, can attain 
improvement in the production process as employees increase cross-categorical communication and improve 
teamwork skills. This approach to placing more emphasis on general knowledge rather than on technical skills, 
may prove beneficial in small companies where most employees perform differentiated tasks. The evaluators 
believe that this basic teaching framework appears to be a model that can be replicated, with adequate training 
of trainers. It is flexible enough to accommodate a variety of content specific skills while emphasizing the 
skills needed by individuals who are required to work in groups or teams with others. Clearly, the approach is 
different from most instructional models, therefore it is essential that others understand what can be 
accomplished through this approach and that they buy into the approach. 

Recommendations 

Assessment and evaluation 

One area of concern that should be addressed by Workplace Literacy Project designers in the need to 
incorporate assessment and evaluation instruments that more closely measure the content of training provided. 
Although the instruments used in this project (ABLE and TALS ) were standardized measures typically used in 
such projects, they are not necessarily designed to measure changes in the concepts that were a major part of 
this training. 

Clarification of Outcomes 

It is recommended that this model or any workplace training provide participants with a clear understanding of 
the project's expectations and what learning outcomes participants can expect to result from their involvement. 
There was a clear change in the expectations expressed by Cycle I participants and Cycle HI participants. Since 
this is a developmental/design project, it is expected that information and activities would improve throughout 
the course of the project. Future use of this model should however provide potential users with clear 
expectations. 

Training for Teachers/Users 

Many of the concepts incorporated in this model (metacognition, facilitated teaching, cooperative learning, etc.) 
are areas which are not commonly incorporated in teacher training programs. Therefore, the model should 
include a clearly defined process for training potential users including resources and references. 
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Please fill in the following information regarding this project: 

1 . Do you think your company should provide classes for courses like this? 
Why? or Why not? 



2. What do you think you will gain from this class? 



3. What are your personal goals for participating in this class? 

4. What kinds of things do you read or write at work? 
read: 



write: 



5. What kinds of things do you read or write when you are not at work? 
read: 



write: 



6. What educational areas (math, reading, writing, communication skills) would you like 
to work on? 



7. What job skills do you need to work on? (e.g. areas you have difficulties with, that 
would increase your productivity level, or help you understand how the system 
operates) 

BEST COPY AVAILABLE 



2 



8. Do you think this class can help you do your job better? 
Why? or Why not? ' 



9. Do you think this class can help improve production levels in your company? 
Why? or Why not? 



10. What jobs of your company do you think should be in this training? 



1 1. Do you use Statistical Process Controls (SPC) on your job? Yes No . 

12. Do you use Total Quality Management (TQM) on your job? Yes No . 

13. Do you like where your class meets? 



14. Is the amount of time spent in class? (check one) 

about right ( ) should be more ( ) should be less ( ) 

15. What job plans do you have for the next few y :ars? (check all that apply) 

( ) promotion ( ) retirement 

( ) start own business ( ) transfer to another company 

( ) transfer to another section within my company 

16. How does more education and training help you with those plans? 
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Workplace Literacy Project 
Partirioantilllllltt^ 



Training Cyde; fit 
__ 
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Please fill in the following information regarding this project: 

1. Do you think your company should provide classes for courses like this? 
Why? or Why not? 



2. What do you think you will learn from this class? 



i 



3. Please check the alternative that better describe your opinion on the following 
statements: 

a. I like learning new concepts 

( ) strongly agree ( ) agree ( ) neutral ( ) disagree ( ) strongly disagree 

b. I enjoy reading on my leisure time 

( ) strongly agree ( ) agree ( ) neutral ( ) disagree ( ) strongly disagree 

c. I read (newspapers, magazines, books, etc.) on a daily basis 

( ) strongly agree ( ) agree ( ) neutral ( ) disagree ( ) strongly disagree 

d. Reading is important to perform my activities at work 

( ) strongly agree ( ) agree ( ) neutral ( ) disagree ( ) strongly disagree 

e. Good writing skills arc necessary to perform my job 

( ) strongly agree ( ) agree ( ) neutral ( ) disagree ( ) strongly disagree 

f. I would like my company to offer more training activities to employees 

( ) strongly agree ( ) agree ( ) neutral ( ) disagree ( ) strongly disagree 



ERIC 



57 



CEST COPY AVAILABLE 



4. Please check all the areas you would like to work on this class? 



reading ' 
teamwork techniques 
writing 

problem solving skills 
communication skills 
basic literacy 

basic skills on other jobs of my company 
statistical process controls (SPC) or equivalent 
total quality management (TQM) 

knowledge of the overall process and activities of my company 
technical skills specifically related to my job description 



5. Please check the alternative that best describe your opinion for each of the following 
statements: 

a. In my opinion, worker to worker communication in my company is... 

( ) very effective ( ) effective ( ) somewhat effective ( ) not effective 

b. In my opinion, worker to supervisor communication in my company is... 

( ) very effective ( ) effective ( ) somewhat effective ( ) not effective 

c. Communication skills are very important to improve work conditions on my job. 
( ) strongly agree ( ) agree ( ) neutral ( ) disagree ( ) strongly disagree 



6. Do you think this class can help you do your job better? 
Why? or Why not? 



7. Do you think this class can help improve production levels in your company? 
Why? or Why not? 
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8. What jobs of your company do you think should be in this training? 



3 



9. What is your definition of quality? 



10. How does your company measure quality? 



1 1. Do you interpret statistical process charts on your job? 
If so, which ones do you use? 



12. In regard to your job, what do you see yourself doing in the next 5 years? 



13. How does more education and training can help you with what you expect to be doing 
in the next 5 years? 



14. Do you like where your class meets? 



i 
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15. Is the amount of time spent in class? (check one) 

about right ( ) should be more ( ) should be less ( ) 
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Workplace Literacy Project 
Post-Traiiiiiifi Participant tSurvev Instrument 


Four-dtfiit number 

; • •••• 


Corapariy/Locatioti: 


Job Clarification: 

i : : ; 


Traiwtog Cycle: I It m 



Please fill in the following information regarding this project: 

1 . Do you think your company should provide classes for courses like this? 
Why? or Why not? 



2. What do you think you gained from this class? 



3. What personal goals did you achieve from participating in this class? 



4. Were the educational areas that you wanted to work on addressed in the class? 



5. What job skills do you still need to work on? (e.g. areas you have difficulties with, that 
would increase your productivity level, or help you understand how the system 
operates) 



i 



6. Identify at least three skills or behaviors that you think have been improved as a result 
from this training. 



i 



7. Were your overall expectations for this class met? 
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8. How could this class better meet your training needs? 



9. Do you think this helped you do your job better? 
Why? or Why not? 



10. Do you think this class helped improve production levels in your company? 
Why? or Why not? 



10. What jobs of your company do you think should be in this training? 



11. Do you have a better understanding of how to use Statistical Process Controls (SPC) 
on your job? Yes No 

12. Do you use Total Quality Management (TQM) on your job? Yes No 



13. Did you like where your class met? 



i 



14. Was the amount of time spent in class? (check one) 

about right ( ) should be more ( ) should be less ( ) 



15. What job plans do you have for the next few years? (check all that apply) 

( ) promotion ( ) retirement 

( ) start own business ( ) transfer to another company 

( ) transfer to another section within my company 
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16. How does this training help you with those plans? 
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1. Do you think the classes will help increase productivity in your section of the 
company? 

Yes ( ) No ( ) 



2. How you think the classes can contribute to increase your company's overall 
productivity? (e.g. by providing cross-training skills, basic work skills, TQM/SPC 
skills, by improving career advancement and job maintenance skills, self-esteem, etc.) 



3. What would be indicators of greater productivity in your section? 



4. What critical skills do the participants in your section of the company need to gain to 
better perform their tasks? 



5. What critical skills do you think the participants from your company will gain from 
their involvement in the literacy program? 
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6. What specific behaviors do you observe in the workers? 





Top worker 


Average worker 


Poor worker 


job classification 








job classification 








job classification 








job classification 
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7. What changes in participants' behavior do you expect to observe as a result of the 
training? 



8. Which of the following type of workers would you like to see participating in the 
program? 

top workers ( ) low performance workers ( ) all workers ( ) 

Why? 

9. How will you measure improvements in participants' performance as a result of the 
program? 



10. What other job classifications do you think would best benefit form this training? 



1 1. How do you think the participants' training schedule (2 hours, 2 days per week) 
affected the productivity rate of the individual? 

Positively ( ) Negatively ( ) 

12. What other opportunities for training does your company provide for employees? 



Workplace Literacy Project 
Pre-lraining Supervisor Survey 



65 



1 



iiiiiii 




Foremem/Su&ervisors Survey Instrument 

' . SSN < last four disks) : '; • : 


Depart*. 

i ■ 




~ - — ■ • - - — " -; 

• : . Trdirtitm -Cycle I . " 

, ,.: Company: Comdyne ' 





1 . Do you think the classes helped increase productivity in your section of the company? 
Yes ( ) No ( ) 

If your answer was "No", go to question 4. 

2. How you think the classes contributed to increase your company's overall 
productivity? (e.g. by providing cross-training skills, basic work skills, TQM/SPC 
skills, by improving career advancement and job maintenance skills, self-esteem, etc.) 



3. What were indicators of greater productivity in your section? 



4. Identify the critical skills that you think the participants from your company gained 
from their involvement in the literacy program? 



Communication skills 

Problem solving skills 

Knowledge of the overall 

Teamwork 

Basic literacy 

Basic skills in other jobs 

Reading 

Math 

SPC 

TQM 
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5. Please check the specific behaviors you have observed in the workers who participated 
in the training and that you believe were positively affected by the training? 



Detailing 



• demonstrates knowledge of the overall process ( ) 

• takes initiative ( ) 

• needs less direction and supervision ( ) 

• demonstrates effective communication skills ( ) 

Sanding 

• demonstrates knowledge of the overall process ( ) 

• minimizes scrap rate ( ) 

• demontrates effective communication skills ( ) 

Pressure Testing 

• demonstrates knolwedge of valve numbers ( ) 

• demonstrates knowledge of PRD numbers ( ) 

• demonstrates knowledge of pressures needed ( ) 

• operates equipment for tests efficiendy ( ) 

• demonstrates basic knowledge ( ) 

• demonstrates knowledge of the overall process ( ) 

• demonstrates effective communication skills ( ) 

Ring Winding 

• demonstrates appropriate reading skills (blueprint) ( ) 

• demonstrates appropriate math skills ( ) 

• demonstrates effective communication skills ( ) 

• demonstrates knowledge of scales (number conversion) ( ) 

Lav Up 

• demonstrates appropriate reading skills (blueprint) ( ) 

• demonstra tes appropriate problem solving skills ( ) 

• demonstrates effective communication skills ( ) 
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6. Do you believe that all workers should be involved in the program? 



7. What measures will you use to determine improvements in participants' performance 
as a result of the program? 



8. How do you think the participants' training schedule (2 hours, 2 days per week) 
affected the productivity rate of the individual? 

Positively ( ) Negatively ( ) 
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1. Do you think the classes helped increase productivity in your section of the company? 
Yes ( ) No ( ) 

If your answer was "No", go to question 4. 

2. How you think the classes contributed to increase your company's overall 
productivity? (e.g. by providing cross-training skills, basic work skills, TQM/SPC 
skills, by improving career advancement and job maintenance skills, self-esteem, etc.) 



3. What were indicators of greater productivity in your section? 



4. Identify the critical skills that you think the participants from your company gained 
from their involvement in the literacy program? 



Communication skills 
Problem solving skills 
Knowledge of the overall 
Teamwork 
Basic literacy 
Basic skills in other jobs 
Reading 
Math 

SPC ( ) 

TQM ( ) 
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5. Please check the specific behaviors you have observed in the workers who participated 
in the training and that you believe were positively affected by the training? 

Die Cuttine 

• demonstrates uderstanding of blueprints ( ) 

• understands and appropriately utilizes job router ( ) 

• demonstrates knowledge of tools and materials ( ) 

• set up time is minimal ( ) 

• demonstrates problem solving skills ( ) 

• practices high safety procedures ( ) 

• uses effective communication skills ( ) 

• minimizes scrap rate ( ) 

Assembly 

• identifies appropriate materials ( ) 

• understands and follows written instructions ( ) 

• practices high safety procedures ( ) 

• performs duties in a timely manner . / ( ) 

• demonstrates ability to perform severaf tasks in the area ( ) 

• provides clearly written communication of problems to engineering ( ) 

• verbal communication to others in group is appropriate and clear ( ) 

• demonstrates problem solving skills ( ) 

• performs with limited or no assistance / supervision ( ) 

Screen Printer 

• set up time is minimal ( ) 

• efficiently matches numbers of job with materials ( ) 

• demonstrates high quality level of inspection ( ) 

• utilizes problem solving skills ( ) 

• practices high safety procedures ( ) 

• uses effective decision making in determination of quality of products ( ) 

• demonstrates ability to analyze problems as appropriate for the task ( ) 

• demonstrates ability to handle complex jobs ( ) 

• scrap rate is reduced ( ) 

Drafts Person 

• demonstrates skills in estimation ( ) 

• translates drawings for tool development ( ) 

• effectively communicates with costumers ( ) 

• utilizes written communication skills effectively ( ) 

• demonstrates knowledge of manufacturing process ( ) 

• demonstrates effective decision making skills ( ) 

• performs quality / accurate / computerized work ( ) 

• needs less supervision ( ) 
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■ 6. Do you believe that all workers should be involved in the program? 

i 

H 7. What measures will you use to determine improvements in participants' performance 

as a result of the program? 



i 



i 

i 
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8. How do you think the participants' training schedule (2 hours, 2 days per week) 
affected the productivity rate of the individual? 

Positively ( ) Negatively ( ) 
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' Company: UwiSysiems] 



1. Do you think the classes helped increase productivity in your section of the company? 
Yes ( ) No ( ) 

If your answer was "No", go to question 4. 

2. How you think the classes contributed to increase your company's overall 
productivity? (e.g. by providing cross-training skills, basic work skills, TQM/SPC 
skills, by improving career advancement and job maintenance skills, self-esteem, etc.) 



3. What were indicators of greater productivity in your section? 



4. Identify the critical skills that you think the participants from your company gained 
from their involvement in the literacy program? 



Communication skills ( ) 

Problem solving skills ( ) 

Knowledge of the overall ( ) 

Teamwork ( ) 

Basic literacy ( ) 

Basic skills in other jobs ( ) 

Reading ( ) 

Math ( ) 

SPC ( ) 

TQM ( ) 
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5. Please check the specific behaviors you have observed in the workers who participated 
in the training and that you believe were positively affected by the training? 



Molder 



demonstrates increased motivation 
requires minimal direction to do job 
demonstrates knowledge of tool preparation 
stays one step ahead on the process 

demonstrates knowledge of the process from beginning to end 
requires minimal time to learn 



Lead Operator 

• demonstrates ability to stay with the task 

• demonstrates knowledge of company ' s goals 

• demonstrates effective verbal and written communication skills 

• demonstrates understanding of quality goals 

• uses problem solving and decision making skills 

• demonstrates knowledge of the process (sequence of tasks) 

• performs tasks with less assistance/supervision 



i 



Value Added 

• demonstrates high motivation ( ) 

• utilizes effective communication skills with vendors, management 

and co-workers ( ) 

• requires less supervision ( ) 

• 6 ;monstrates appropriate level of performance ( ) 

• demonstrates understanding of the process ( ) 




6. Do you believe that all workers should be involved in the program? 



7. What measures will you use to determine improvements in participants' performance 
as a result of the program? 



8. How do you think the participants' training schedule (2 hours, 2 days per week) 
affected the productivity rate of the individual? 

Positively ( ) Negatively ( ) 
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1 . Do you think the classes helped increase productivity in your section of the company? 
Yes ( ) No ( ) 

If your answer was "No", go to question 4. 

2. How you think the classes contributed to increase your company's overall 
productivity? (e.g. by providing cross-training skills, basic work skills, TQM/SPC 
skills, by improving career advancement and job maintenance skills, self-esteem, etc.) 



3. What were indicators of greater productivity in your section? 



4. Identify the critical skills that you think the participants from your company gained 
from their involvement in the literacy program? 



Communication skills 

Problem solving skills 

Knowledge of the overall 

Teamwork 

Basic literacy 

Basic skills in other jobs 

Reading 

Math 

SPC 

TQM 
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5. Please check the specific behaviors you have observed in the workers who participated 
in the training and that you believe were positively affected by the training? 



Contract Assistant 



• performs functions with less/little direction 

• willing to take control, learn new skills 

• needs less assistance from peers 



( ) 
( ) 
( ) 



Contract Administration 

• makes decisions appropriately 

• improved written/verbal communication skills 

• takes initiative 

• needs less direction and supervision 



( ) 
( ) 
( ) 
( ) 



i 



i 
i 
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Senior Business Administrator 

• demonstrates analytical thought process 

• conceptualizes easily 

• makes appropriate decisions 

• demonstrates ability to plan and implement strategies 

• demonstrates vision of the whole picture 

• takes initiative 

• has shown willingness to expand knowledge 



Team Member 

• takes initiative 

• assists others 

• willing to volunteer, go beyond what is expected 

• communication skills 
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6. Do you believe that all workers should be involved in the program? 



|j 1. What measures will you use to determine improvements in participants' performance 

as a result of the program? 



i 
i 

E 8. How do you think the participants' training schedule (2 hours, 2 days per week) 

affected the productivity rate of the individual? 

S Positively ( ) Negatively ( ) 
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Name '■• :i ^BiBlIl8i-.. -iSMl 
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Date: ________ _ Company: . Grime* ? Twain Ave. 



1 . Do you think the classes helped increase productivity in your section of the company? 
Yes ( ) No ( ) 

If your answer was "No", go to question 4. 

2. How you think the classes contributed to increase your company's overall 
productivity? (e.g. by providing cross-training skills, basic work skills, TQM/SPC 
skills, by improving career advancement and job maintenance skills, self-esteem, etc.) 



3. What were indicators of greater productivity in your section? 



4. Identify the critical skills that you think the participants from your company gained 
from their involvement in the literacy program? 



Communication skills ( 

Problem solving skills ( 

Knowledge of the overall ( 

Teamwork ( 

Basic literacy ( 

Basic skills in other jobs ( 

Reading ( 

Math ( 

SPC ( 

TQM ( 
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5. Please check the specific behaviors you have observed in the workers who participated 
in the training and that you believe were positively affected by the training? 



Analyst * 

• efficiency levels 

• utilization levels 

• analysis time 

• utilizes problem solving skills 

• demonstrates ability to work in teams 

• demonstrates leadership 

• requires less supervision from co-workers / supervisors 

• communication skills are effective 

Repair Technician 

• efficiency levels 

• utilization levels 

• analysis time 

• demonstrates interpretation skills 

• demonstrates knowledge of quality control 

• utilizes effective decision making skills 

• demonstrates problem solving skills 

• demonstrates ability to work in teams 

• requires less supervision from co-workers 

• communication skills are effective 

Inspector 

• appropriately interprets regulations 

• effectively applies regulations (Federal and customer requests) 

• demonstrates initiative 

• demonstrates ability to work iu teams 

• efficiency levels 

• utilization levels 

• demonstrates understanding of SPC/TQM 
Receiving 

• demonstrates appropriate reading skills 

• demonstrates appropriate typing skills 

• communication skills are effective 

• uses question & answer appropriately 

• demonstrates interpretation skills 

• demonstrates ability to work in teams 

• demonstrates memory skills 

• problem solves effectively 
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6. Do you believe that all workers should be involved in the program? 



7. What measures will you use to determine improvements in participants * performance 
as a result of the program? 



8. How do you think the participants* training schedule (2 hours, 2 days per week) 
affected the productivity rate of the individual? 

Positively ( ) Negatively ( ) 



Workplace Literacy Project 
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C. Interview protocols 
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admin. Interview 



1. What is the current organizational structure for the project? 

2. What are the roles of key project staff? 

3. How will this reorganization benefit the project? 

4. How are support resources (technical resources) being used in the overall 
management and implementation of the project? 

5. What progress is being made towards the development and documentation of 
the training model? 

what mechanisms are being used to document the training model? 

how accurate are the records? 

how comprehensive is the information? 

6. How much effort is being devoted toward planning activities in relation to 
class content, processes, materials, teacher feedback? (% of time) 

7. What changes have been implemented (specific to cycles II & III) in the 
training process, and communication systems among project staff and 
between project staff and companies? 

8. Do you think the project is accomplishing the stated objectives? Explain. 

9. Does the content of this training contribute to establishing a high productivity 
environment? Yes, no, explain. 

10. Does the content of this training contribute to job maintenance and or career 
advancement? Yes, no, explain 

1 1 . Does the training curriculum improved basic work skills and TQM/SPC 
processes? Yes, no, explain 

12. Does the training curriculum contribute to workplace skills development? 
Yes, no, explain 

13. What activities are being implemented to continue the provision of services to 
existing partners and to other business in the county? 
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14. What have you learned from this implementation that would tell you where 
this model could better be applied? 

15. What contributions has the problem solving committee made to the model? 

16. What would you change in terms of project implementation, curriculum and 
process? 
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Teacher/Facilitator Interview 



B 



B 



I - Class Preparation 

1. What activities have you engaged in to develop/strengthen your skills as 
teacher/facilitator of classes? (between cycles I & II and II & III) 

2. What do you perceive are your strengths/weaknesses now as opposed to 
initially? (how this changed from cycle I to present) 

3. What do you do conceptually to prepare for classes? 

4. What specific methodology is used to impact knowledge? (lecture, 
demonstration, small group activities, etc.) 

5. What is the source of materials used in class? (workplace materials, 
commercial, self-developed) 



i 

i 
i 



i 
i 

i 



H - Classroom activities/Implementation 

6. What are the major content foci for the class? Is it the same for all companies? 
What are the common content strands across companies and what are specific 
content strands for each site? 

7. To what extent TQM/SPC are a part of the curriculum? (if it was changed, 
what were the reasons for such changes?) 



8. How decisions are made in order to develop a sequence to which concepts are 
presented? 

9. Is there a consistent format to develop lesson plans? Objectives are 
individualized or follow a large group focus? 

10. What is the class framework? (example of class structure) 
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1 1. How much of overall class time is spent on each concept/application? Indicate 
how job responsibilities of the participants determine the emphasis given to 
concept/applications. 

• Check the sites to which any concept/application is used more frequently or 
given more emphasis. 

• Indicate your perceptions on applicability of concepts/applications to the 
different worksites (check the ones where information is more 
applicable/helpful) 



i 



Concept 



Problem Solving 



Learning styles 



Communications (verbal) 



Communications (written) 



Overall 



Grimes 



Grimes 
(Rt.55) 



Grimes 
(CBT) 



Hall 



Comdyne 



Lewis 



Application 


Overall 


Grimes 
(TA) 


Grimes 
(Rt.55) 


Grimes 
(CBT) 


Hall 


Comdyne 


Lewis 


Lecture 




























Demonstration 




























Practice 




























Small group activities 




























Workplace examples 

















































































































12. Under what conditions are unplanned activities/concepts used? How frequent 
does this happen? How much time do you spend on activities not covered in 
the lesson plan? 

13. What other concepts need to be a part of this model based on any feedback you 
may have obtained from participants? 



0 

ERLC 



Workplace Literacy Project 
Project Staff Interview - 06/15/94 



85 



III. Informal Evaluation 
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14. How do you measure learning outcomes? How do you know when learning 
has been achieved? (considering standardized tests don't measure learning 
outcomes) 



15, How do you use this information to design future lesson plans? 



16. What environmental factors (site, project, and participant related) have 
impacted your classes and in what way? 

17. What is your overall perception of the utility/applicability of the 
information/knowledge given by this project to the participant companies. 



IV - Overall Project Management 



18. Do you think the project is accomplishing the stated objectives? Yes, no, 
explain. 

19. Does the content of this training contribute to establishing a high productivity 
environment? Yes, no, explain. 

20. Does the content of this training contribute to job maintenance and or career 
advancement? Yes, no, explain. 

21. Does the training curriculum improved basic work skills and TQM/SPC 
processes? Yes, no, explain. 

22. Does the training curriculum contribute to workplace skills development? Yes, 
no, explain 

23. What activities are being implemented to continue the provision of services to 
existing partners and to other business in the county? 

24. How much time has project staff spent on helping on your perceived needs to 
improve what you are doing? (assistance, training, etc.) 
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8 25. Have there been any issues from sites/supervisors or management? Kow these 

issues have been handled by project staff? 

I 

26. Are there any areas of this project that you would recommend changing? What 
g areas? How? 

i 

I 

i 
i 
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Foremen Interview 

1. What is your overall perception of the effectiveness of the project? 

Did you participate on supervisors meetings with project staff? 

Do you believe that the project is accomplishing its goals and objectives? 

2. What is your sense of what the participants are learning? 

3. Have you observed that any participants from your site have applied what they 
learned? 

In what way? 
problem solving 
communication 
job task related skills 

4. Describe any noticeable changes in productivity that you have observed in 
employees who participated in this training cycle 

5. Do you feel that gains made by participant merits continuation of this type of 
learning opportunity? 

6. Do you feel that any follow up on extended learning opportunity should be 
made available to employees? 

Do you feel that a similar learning opportunity should be made available for 
foremen/supervisors and other company managers? 

7 . Comments/suggestions 
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D. Productivity 
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Workplace Literacy Project 
Comdvne 



Participants Productivity Data ' 
General information on what data to be collected: 

Pre and post data: Baseline data has to be collected from period just before classes 
started Post class data should be collect from a period of at least a month after each cycle 
of classes was over. 

Sample size: at least 30 observations on each variable for each participant (e.g. 30 
individual time charts, one month of timeliness information - i.e. late days, absenteeism, 
etc.). 

Representativeness of the sample: sample has to cover periods that are similar to all 
workers and that are similar for baseline and after class information. That is, 
participant data has to be consistent (same time of the day, or the entire same month for ail 
workers) 

Individual data: all data has to be collected for every participant, individually. 
Arrangements will have to be made in the cases were mdividual information is not 
available. Such arrangements could be to allow project evaluators to measure 
performance; to engage participant team in problem solving to find ways to obtain the 
data; to obtain information on how many members of a job classification (or how many job 
classifications) participated in the project and how many members (or job classifications) 



did not. 




Job OassSficatimt . 

• 




Detailing 


• Timeliness 




• Daily production 




• Indicators of quality 


Sanding 


• Timeliness 




• Daily production 




• Scrap rate 


Pressure Testing (valvers) 


• Timeliness 




• Daily production 


Ring Winding (winders) 


• Timeliness (attendance) 




• Daily production 




• Scrap rate 


Lay Up 


• Timeliness 




• Daily production 



If you have any questions, please contact Cesar Dagord at (614) 447-0844, FAX (614) 447-9043 
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Data Collection Sheet 
Project Participants Productivity Data 
Comdyne 



Job Classification: 

Detailing 
Sanding 
Valvers 
Winders 
Lay Up 



Cycle: 

( ) Cycle I 
( ) Cycle H 
( ) Cycle HI 



Period: 

( ) pre-training 
( ) post-training 



ID number 



(4 digit) 



Date 


Timeliness 

(attendance, 
days early/late) 


Scrap rate 


Daily 
Production 


Indicators 
of Quality 


Other 

(specify) 
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Workplace Literacy Project 
Grimes - CBT 

Parf icinants Productivity Data 

General information on what data to be collected: 

Pre and post data'. Baseline data has to be collected from period just before classes 
started. Post class data should be collect from a period of at least a month after each cycle 
of classes was over. 

Sample size: at least 30 observations on each variable for each participant (e.g. 30 
individual time charts, one month of timeliness information - i.e. late days, absenteeism, 
etc.). 

Representativeness of the sample: sample has to cover periods that are similar to all 
workers and that are similar for baseline and after class information. That is, 
participant data has to be consistent (same time of the day, or the entire same month for all 
workers) 

Individual data: all data has to be collected for every participant, individually. 
Arrangements will have to be made in the cases were individual information is not 
available. Such arrangements could be to allow project evaluators to measure 
performance; to engage participant team in problem solving to find ways to obtain the 
data; to obtain information on how many members of a job classification (or how many job 
classifications) participated in the project and how many members (or job classifications) 
did not. 



Job Gasification 




Contract Assistant 




Contract Administration 




Senior Business Administrator 




Team Member 





If you have any questions, please contact Cesar Dagord at (614) 447-0844, FAX (614) 447-9043 
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Data Collection Sheet 
Project Participants Productivity Data 
Grimes - CBT 

Job Classification: Cycle: 

( ) Contract Assistant ( ) Cycle I 

( ) Contract Administrator ( ) Cycle II 
( ) Sr. Business Administrator ( ) Cycle III 
( ) Team member 

Indicate productivity indicator that is consistent for pre- and post- periods and within 



the period colli 


tctedL 




Date 


Productivity 
indicator 1: 


Productivity 
indicator 2: 


Productivity 
indicator 3: 


Other 

(specify) 


Other 

(specify) 
















1 


































. — 



























































































































































































































































































































Period: D) number 

( ) pre-training 

( ) post-training (4 digit) 
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Workplace Literacy Project 
Grimes - Twain Ave. 

Participants Prod uctivity Data 

General information on what data to be collected: 

Pre and post data'. Baseline data has to be collected from period just before classes 
started. Post class data should be collect from a period of at least a month after each cycle 
of classes was over. 

Sample size: at least 30 observations on each variable for each participant (e.g. 30 
individual time charts, one month of timeliness information - i.e. late days, absenteeism, 
etc.). 

Representativeness of the sample: sample has to cover periods that are similar to all 
workers and that are similar for baseline and after class information. That is, 
participant data has to be consistent (same time of the day, or the entire same month for all 
workers) 

Individual data: all data has to be collected for every participant, individually. 
Arrangements will have to be made in the cases were individual information is not 
available. Such arrangements could be to allow project evaluators to measure 
performance; to engage participant team in problem solving to find ways to obtain the 
data; to obtain information on how many members of a job classification (or how many job 
classifications) participated in the project and how many members (or job classifications) 
did not 



Analyst 


Sagged dau to collected J 

• Efficiency level 

• Utilization levels 

• Analysis time 


Repair 
Technician 


• Efficiency level 

• Utilization levels 

• Analysis time 


Inspector 


• Efficiency 

• Utilization 

• Standardized quantitative tests (obtain 4-digit numbers of these 
participants to use quantitative tests as indicator of productivity) 


Receiving 


• Use battery of standardized tests (obtain 4-digit numbers of these 
participants and use standardized tests as productivity index) 
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Data Collection Sheet 
Project Participants Productivity Data 
Grimes • Twain Ave. 



i 
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Job Classification: 

( ) Analyst 

( ) Repair Technician 

( ) Inspector 



Cycle: 
( ) Cycle I 
( ) Cycle H 
( ) Cycle HI 



Period: 

( ) pre-training 
( ) post-training 



ID number 



(4 digit) 



i 



i 
i 



Date 


Efficiency 
Level 


Utilization 
Level 


Analysis 1 
Time 


Other 

(specify) 


utner j 

(.specify; j 
































































































































1 
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Workplace Literacy Project 
Hall Company 



y Participants Productivity Data 

General information on what data to be collected 

Pre and post data: Baseline data has to be collected from period just before classes 
started Post class data should be collect from a period of at least a month after each cycle 
of classes was over. 

Sample size: at least 30 observations on each variable for each partic ; pant (e.g. 30 
individual time charts, one month of timeliness information - i.e. late days, absenteeism, 
etc.). 

Representativeness of the sample: sample has to cover periods that are similar to all 
workers and that are similar for baseline and after class information. That is, 
participant data has to be consistent (same time of the day, or the entire same month for all 
workers) 

Individual data: all data has to be collected for every participant, individually. 
Arrangements will have to be made in the cases were individual information is not 
available. Such arrangements could be to allow project evaluators to measure 
performance; to engage participant team in problem solving to find ways to obtain the 
data; to obtain information on how many members of a job classification (or how many job 
classifications) participated in the project and how many members (or job classifications) 
did not. 





Suggested <ktat* be effected 


Cutters 




• Timeliness (average days early or late) 

• Scrap rate report 

• Percent of jobs on time 

• Productivity (actual hours v. planned hours) 


Assemblers 


• Timeliness (average days early or late) 

• Scrap rate report 

• Percent of jobs on time 

• Productivity (actual hours v. planned hours) 


Screen Printers 


• Timeliness (average days early or late) 

• Scrap rate report 

• Percent of jobs on time 

• Productivity (actual hours v. planned hours) 


Drafts Person 


• Timeliness (average days early or late) 

• Percent of jobs on time 1 



If you have any questions, please contact Cesar Dagord at (614) 447*0844, FAX (614) 447-9043 
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Data Collection Sheet 
Project Participants Productivity Data 
Hall Company 

Job Classification: Cycle: Period: ID number 

( ) Cutter ( ) Cycle I ( ) pre-training 

( ) Assembler ( ) Cycle II ( ) post-training (4 digit) 

( ) Drafts Person ( ) Cycle III 

( ) Screen Printer 



Date 


Timeliness 

(average 

Uajro CiUiy/lalC / 


Scrap Rate 


Productivity 

(actual hours v. 
claimed hours^ 


Percent of 
Jobs on Time 


Other 

(specify) 












... 
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Workplace Literacy Project 
Lewis Systems 

Participants Productivity Data 

General information on what data to be collected: 

Pre and post data: baseline data has to be collected from a period just before classes 
started Post class data should be collect from a period of at least a month after each cycle 
of classes was over. 

Sample size: at least 30 observations on each variable for each participant (e.g. 30 
individual time charts, one month of timeliness information - i.e. late days, absenteeism, 
etc.). 

Representativeness of the sample: sample has to cover periods that are similar to all 
workers and that are similar for baseline and after class information. That is, 
participant data has to be consistent (same time of the day, or the entire same month for all 
workers, etc.) 

Individual data: all data has to be collected for every participant, individually. 
Arrangements will have to be made in the cases were individual information is not 
available. Such arrangements could be to allow project evaluators to measure 
performance; to engage participant team in problem solving to find ways to obtain the 
data; to obtain information on how many members of a job classification (or how many job 
classifications) participated in the project and how many members (or job classifications) 
did not 



I 
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Job Classification 


Suggested cteta- to he collected " 


Maintenance 


• Observation of paper work activities: data could 
be a sample (pre/post classes) of the paper work. 


Press Operators 


Graph charts (time charted for each worker). 
Collect graphs from before and after classes were 
taken. 


Molder 


• Reading skills: obtain 4-digit number of these 
participants: will use standardized tests as a 
substitute for "productivity" data for these 
participants. 


Secondary 


Time spent on each job. Data can best be 
obtained by observation (closest supervisor of 
worker?). 



If you have any questions, please contact Cesar Dagord at (614) 447-0844, FAX (614) 447-9043 
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Data Collection Sheet 
Project Participants Productivity Data 
LewiSystems 

Job Classification: Cycle: Period: ID number 

( ) Press Operators ( ) Cycle I ( ) pre-training 

( ) Secondary ( ) Cycle II ( ) post-training (4 digit) 

( ) Cycle HI 



Date 


Graph 
Charts 

(time charted) 


Time spent 
on each job 

(observation) 


Other 
Indicators of 
Quality 

/ £*f"4/3t/"» 1 ft \ M 

VSpcClIICyj 


Other 

(specify) 


Other 

(specify) 
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Workplace Literacy Curriculum 
Explanation Development Thematic Units/Lesson Plan 

Explanation 

This project was bawd on neuropsychological brain behavior research that indicates that all 
people can Icara best in a complex, natural environment Ncuroscicncc identifies 
differences in construction of memory and methods of building knowledge systems. The 
brain learns through patterns, feelings, wants, choice, reasoning, moral values and other 
"things of the mind" (Roger Speny, 1986, Nobel Prize Laureate responsible for the initial 
"split brain* research). Learning is equated with change in that the brain stores patterns 
with complex connections of such quality and control as to be accessed appropriately in 
new learning situations. 

In the past, learning specific facts to be remembered and produced in predictable situations 
was adequate. Today, global economic concerns demand employees who demonstrate a 
high level of competence to deal with complexity and change. This demand dictates that 
the traditional methods of linear, teacher-directed training be allowed to change. The 
naturally complex work environment provides excellent content for cognitive, student- 
directed learning. 

Curriculum suited to the workplace needs a marriage of information and situations 
available in the worlq>lace with cognitive thinking and cooperative education methods. The 
teacher/coach and die class participants monitor the content and the process of learning. 
The participants gain a sense of control over learning that translates to all part of life. 

Development 

The process of developing the Woikplacc grant curriculum involved the following steps: 
♦identify and develop thematic units 
♦selection of units appropriate to each site 

♦specific de sign of lesson plans for each site, job classification and individual 
learner 

The Problem Solving Committee provided the expertise to identify topics relevant to 
current and future business needs. The committee of company executives, class 
participants aad the Workplace Team were able to collaborate on these issues. The topics 
chosen by this group were Cycle Time, Statistical Processing Control, Quality Issues and 
skilk, 

The topics were developed into thematic units by the Workplace Team. The content, 
thinking and social/communication skills of each unit were identified and prioritized 

Customizing the curriculum for the particular site, job classification and individual learner 
involved numerous steps. Company personnel on the Problem Solving Committee 
prioritized which units were to be taught at each site. The Workplace Team conducted 
informal Literacy Task Analysis of each new job classification to familiarize themselves 
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with the job demands. Individual lesson plans were initially designed from the above 
information and (he results of the standardized pretests. Class participants completed 
literacy Task Analysis of their own job and this information was also used to direct the 
lesson planning. Most of the classroom content was derived from workplace materials. 
Linear, prcdcvclopcd worksheets and exercises were never unixEzed in this curriculum. 

The format for the lesson plan development was consistent throughout the above process, 
The content, thinking and social/commun5cation skills involved in each substdQ were 
identified on the the thematic units, the Team LTA's and the participant LTA's. The 
information was gathered from company executives, class participants, and the Workplace 
Team. 



Lesson Plans/Classroom Management 

Each lesson was introduced with mctacognitrve awareness and strategy activities. 
Application of these sldQs was then included in cooperative and discovery learning activities 
based on common life experiences. The final part of e*ch lesson involved transfer of these 
skills to the workplace through additional cooperative education activities. 

During the course of the teaching cycle, less time was devoted to learning mctacognitrve 
skills and strategies and more time was spent on application. Teachers were prepared to 
direct teach only an average of 20 minutes and to use facilitator/coaching skills for the 
remainder of the 2 hour class session. 



Elizabeth will be elaborating on each detail of this summary. She will also show a 
breakdown of the subsldHs involved in each of the thematic units. 
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